tokenizer_state Struct Reference

#include <linguisticProcessing/core/FlatTokenizer/SpiritTokenizerParser.hpp>

Public Attributes
std::string	id
std::vector< tokenizer_transition >	transitions

Detailed Description

Definition at line 45 of file SpiritTokenizerParser.hpp.

Member Data Documentation

id

std::string tokenizer_state::id

Definition at line 47 of file SpiritTokenizerParser.hpp.

transitions

std::vector<tokenizer_transition> tokenizer_state::transitions

Definition at line 48 of file SpiritTokenizerParser.hpp.

---

The documentation for this struct was generated from the following file:

• lima_linguisticprocessing/src/linguisticProcessing/core/FlatTokenizer/SpiritTokenizerParser.hpp