Struct ChainedTokenizerComponent<TChar>

Namespace

Farkle.Parser.Tokenizers

Assembly

Farkle.dll

Represents a component of a chained tokenizer.

public readonly struct ChainedTokenizerComponent<TChar>

Type Parameters

TChar

The type of characters the tokenizer reads.

Inherited Members

ValueType.Equals(object)

ValueType.GetHashCode()

ValueType.ToString()

Properties

Default

A ChainedTokenizerComponent<TChar> that represents the default tokenizer.

public static ChainedTokenizerComponent<TChar> Default { get; }

Property Value

ChainedTokenizerComponent<TChar>

Remarks

When passing the builder to WithTokenizerChain(params ReadOnlySpan<ChainedTokenizerComponent<char>>), the default tokenizer is the parser's existing tokenizer. Otherwise it is the tokenizer specified in CreateChain<TChar>(ReadOnlySpan<ChainedTokenizerComponent<TChar>>, IGrammarProvider?, Tokenizer<TChar>?), if provided.

Methods

Create(Tokenizer<TChar>)

Creates a ChainedTokenizerComponent<TChar> from a Tokenizer<TChar>.

public static ChainedTokenizerComponent<TChar> Create(Tokenizer<TChar> tokenizer)

Parameters

tokenizer Tokenizer<TChar>

The tokenizer to use.

Returns

ChainedTokenizerComponent<TChar>

Create(Func<IGrammarProvider, Tokenizer<TChar>>)

Creates a ChainedTokenizerComponent<TChar> from a tokenizer factory.

public static ChainedTokenizerComponent<TChar> Create(Func<IGrammarProvider, Tokenizer<TChar>> tokenizerFactory)

Parameters

tokenizerFactory Func<IGrammarProvider, Tokenizer<TChar>>

A delegate that creates a Tokenizer<TChar> from an IGrammarProvider.

Returns

ChainedTokenizerComponent<TChar>

Operators

implicit operator ChainedTokenizerComponent<TChar>(Tokenizer<TChar>)

Implicit conversion operator from Tokenizer<TChar>

public static implicit operator ChainedTokenizerComponent<TChar>(Tokenizer<TChar> tokenizer)

Parameters

tokenizer Tokenizer<TChar>

The tokenizer.

Returns

ChainedTokenizerComponent<TChar>

See Also

Create(Tokenizer<TChar>)