Top Description Fields Constructors Methods
org.antlr.v4.runtime

public Class CommonTokenStream

extends BufferedTokenStream
Class Inheritance
Imports
org.antlr.v4.runtime.misc.NotNull
This class extends BufferedTokenStream with functionality to filter token streams to tokens on a particular channel (tokens where Token#getChannel returns a particular value).

This token stream provides access to all tokens by index or when calling methods like getText. The channel filtering is only used for code accessing tokens via the lookahead methods LA, LT, and LB.

By default, tokens are placed on the default channel (Token#DEFAULT_CHANNEL), but may be reassigned by using the ->channel(HIDDEN) lexer command, or by using an embedded action to call Lexer#setChannel.

Note

lexer rules which use the ->skip lexer command or call Lexer#skip do not produce tokens at all, so input text matched by such a rule will not be available as part of the token stream, regardless of channel.

Field Summary

Modifier and TypeField and Description
protected int
channel

Specifies the channel to use for filtering tokens.

Inherited from org.antlr.v4.runtime.BufferedTokenStream:
fetchedEOFptokenstokenSource

Constructor Summary

AccessConstructor and Description
public
CommonTokenStream(TokenSource
The token source.
tokenSource)

Constructs a new CommonTokenStream using the specified token source and the default token channel (Token#DEFAULT_CHANNEL).

public
CommonTokenStream(TokenSource
The token source.
tokenSource, int
The channel to use for filtering tokens.
channel)

Constructs a new CommonTokenStream using the specified token source and filtering tokens to the specified channel.

Method Summary

Modifier and TypeMethod and Description
protected int
adjustSeekIndex(int
The target token index.
i)

Overrides org.antlr.v4.runtime.BufferedTokenStream.adjustSeekIndex.

Allowed derived classes to modify the behavior of operations which change the current stream position by adjusting the target token index of a seek operation.
public int
getNumberOfOnChannelTokens()

Count EOF just once.

protected Token
LB(int k)

Overrides org.antlr.v4.runtime.BufferedTokenStream.LB.

public Token
LT(int k)

Overrides org.antlr.v4.runtime.BufferedTokenStream.LT.

Implements org.antlr.v4.runtime.TokenStream.LT.

Get the Token instance associated with the value returned by LA(k).
Inherited from org.antlr.v4.runtime.BufferedTokenStream:
consumefetchfillfilterForChannelgetgetgetHiddenTokensToLeftgetHiddenTokensToLeftgetHiddenTokensToRightgetHiddenTokensToRightgetSourceNamegetTextgetTextgetTextgetTextgetTokensgetTokensgetTokensgetTokensgetTokenSourceindexLAlazyInitmarknextTokenOnChannelpreviousTokenOnChannelreleaseresetseeksetTokenSourcesetupsizesync

Field Detail

channelback to summary
protected int channel

Specifies the channel to use for filtering tokens.

The default value is Token#DEFAULT_CHANNEL, which matches the default channel assigned to tokens created by the lexer.

Constructor Detail

CommonTokenStreamback to summary
public CommonTokenStream(TokenSource tokenSource)

Constructs a new CommonTokenStream using the specified token source and the default token channel (Token#DEFAULT_CHANNEL).

Parameters
tokenSource:TokenSource

The token source.

CommonTokenStreamback to summary
public CommonTokenStream(TokenSource tokenSource, int channel)

Constructs a new CommonTokenStream using the specified token source and filtering tokens to the specified channel. Only tokens whose Token#getChannel matches channel or have the Token#getType equal to Token#EOF will be returned by the token stream lookahead methods.

Parameters
tokenSource:TokenSource

The token source.

channel:int

The channel to use for filtering tokens.

Method Detail

adjustSeekIndexback to summary
protected int adjustSeekIndex(int i)

Overrides org.antlr.v4.runtime.BufferedTokenStream.adjustSeekIndex.

Doc from org.antlr.v4.runtime.BufferedTokenStream.adjustSeekIndex.

Allowed derived classes to modify the behavior of operations which change the current stream position by adjusting the target token index of a seek operation. The default implementation simply returns i. If an exception is thrown in this method, the current stream index should not be changed.

For example, CommonTokenStream overrides this method to ensure that the seek target is always an on-channel token.

Parameters
i:int

The target token index.

Returns:int

The adjusted target token index.

Annotations
@Override
getNumberOfOnChannelTokensback to summary
public int getNumberOfOnChannelTokens()

Count EOF just once.

LBback to summary
protected Token LB(int k)

Overrides org.antlr.v4.runtime.BufferedTokenStream.LB.

Annotations
@Override
LTback to summary
public Token LT(int k)

Overrides org.antlr.v4.runtime.BufferedTokenStream.LT.

Implements org.antlr.v4.runtime.TokenStream.LT.

Doc from org.antlr.v4.runtime.TokenStream.LT.

Get the Token instance associated with the value returned by LA(k). This method has the same pre- and post-conditions as IntStream#LA. In addition, when the preconditions of this method are met, the return value is non-null and the value of LT(k).getType()==LA(k).

Annotations
@Override