ANTLR 4 Runtime 4.13.1 · org.antlr.v4.runtime.CommonTokenFactory

Field Summary

Modifier and Type	Field and Description
protected final boolean	copyText Indicates whether `CommonToken#setText` should be called after constructing tokens to explicitly set the text.
public static final TokenFactory<CommonToken>	DEFAULT The default `CommonTokenFactory` instance.

Constructor Summary

Access	Constructor and Description
public	CommonTokenFactory(boolean The value for `copyText`. copyText) Constructs a `CommonTokenFactory` with the specified value for `copyText`.
public	CommonTokenFactory() Constructs a `CommonTokenFactory` with `copyText` set to `false`.

Method Summary

Modifier and Type	Method and Description
public CommonToken	create(Pair<TokenSource, CharStream> source, int type, String text, int channel, int start, int stop, int line, int charPositionInLine) Implements org.antlr.v4.runtime.TokenFactory.create. This is the method used to create tokens in the lexer and in the error handling strategy.
public CommonToken	create(int type, String text) Implements org.antlr.v4.runtime.TokenFactory.create. Generically useful

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

Field Detail

copyText back to summary

copyText	back to summary
protected final boolean copyText Indicates whether `CommonToken#setText` should be called after constructing tokens to explicitly set the text. This is useful for cases where the input stream might not be able to provide arbitrary substrings of text from the input after the lexer creates a token (e.g. the implementation of `CharStream#getText` in `UnbufferedCharStream` throws an `UnsupportedOperationException`). Explicitly setting the token text allows `Token#getText` to be called at any time regardless of the input stream implementation. The default value is `false` to avoid the performance and memory overhead of copying text for every token unless explicitly requested.

protected final boolean copyText

Indicates whether CommonToken#setText should be called after constructing tokens to explicitly set the text. This is useful for cases where the input stream might not be able to provide arbitrary substrings of text from the input after the lexer creates a token (e.g. the implementation of CharStream#getText in UnbufferedCharStream throws an UnsupportedOperationException). Explicitly setting the token text allows Token#getText to be called at any time regardless of the input stream implementation.

The default value is false to avoid the performance and memory overhead of copying text for every token unless explicitly requested.

DEFAULT back to summary

DEFAULT	back to summary
public static final TokenFactory<CommonToken> DEFAULT The default `CommonTokenFactory` instance. This token factory does not explicitly copy token text when constructing tokens.

public static final TokenFactory<CommonToken> DEFAULT

The default CommonTokenFactory instance.

This token factory does not explicitly copy token text when constructing tokens.

Constructor Detail

CommonTokenFactory back to summary

CommonTokenFactory	back to summary
public CommonTokenFactory(boolean copyText) Constructs a `CommonTokenFactory` with the specified value for `copyText`. When `copyText` is `false`, the `DEFAULT` instance should be used instead of constructing a new instance. Parameters copyText:boolean The value for `copyText`.

public CommonTokenFactory(boolean copyText)

Constructs a CommonTokenFactory with the specified value for copyText.

When copyText is false, the DEFAULT instance should be used instead of constructing a new instance.

Parameters

copyText:boolean: The value for copyText.

CommonTokenFactory back to summary

CommonTokenFactory	back to summary
public CommonTokenFactory() Constructs a `CommonTokenFactory` with `copyText` set to `false`. The `DEFAULT` instance should be used instead of calling this directly.

public CommonTokenFactory()

Constructs a CommonTokenFactory with copyText set to false.

The DEFAULT instance should be used instead of calling this directly.

Method Detail

create	back to summary
public CommonToken create(Pair<TokenSource, CharStream> source, int type, String text, int channel, int start, int stop, int line, int charPositionInLine) Implements org.antlr.v4.runtime.TokenFactory.create. Doc from org.antlr.v4.runtime.TokenFactory.create. This is the method used to create tokens in the lexer and in the error handling strategy. If text!=null, than the start and stop positions are wiped to -1 in the text override is set in the CommonToken. Annotations @Override

create

back to summary

public CommonToken create(Pair<TokenSource, CharStream> source, int type, String text, int channel, int start, int stop, int line, int charPositionInLine)

Implements org.antlr.v4.runtime.TokenFactory.create.

Doc from org.antlr.v4.runtime.TokenFactory.create.

This is the method used to create tokens in the lexer and in the error handling strategy. If text!=null, than the start and stop positions are wiped to -1 in the text override is set in the CommonToken.

Annotations

@Override

create	back to summary
public CommonToken create(int type, String text) Implements org.antlr.v4.runtime.TokenFactory.create. Doc from org.antlr.v4.runtime.TokenFactory.create. Generically useful Annotations @Override