Namespace Farkle.Parser.Tokenizers
Classes
- ParserInputReaderTokenizerExtensions
Provides extension methods on ParserInputReader<TChar> specific to tokenizers.
- Tokenizer
Provides factory methods to create Tokenizer<TChar>s.
- Tokenizer<TChar>
Provides an interface to break a sequence of characters into tokens.
Structs
- ChainedTokenizerComponent<TChar>
Represents a component of a chained tokenizer.
- TokenizerResult
Represents a Tokenizer<TChar>'s successful or failed result.
Interfaces
- ITokenizerResumptionPoint<TChar, TArg>
Provides a way for a Tokenizer<TChar> to resume at multiple points after suspending.