Top Description Fields Constructors Methods
org.antlr.runtime

public Class CommonTokenStream

extends BufferedTokenStream
Class Inheritance
Known Direct Subclasses
org.antlr.runtime.TokenRewriteStream

The most common stream of tokens where every token is buffered up and tokens are filtered for a certain channel (the parser will only see these tokens). Even though it buffers all of the tokens, this token stream pulls tokens from the tokens source on demand. In other words, until you ask for a token using consume(), LT(), etc. the stream does not pull from the lexer. The only difference between this stream and BufferedTokenStream superclass is that this stream knows how to ignore off channel tokens. There may be a performance advantage to using the superclass if you don't pass whitespace and comments etc. to the parser on a hidden channel (i.e., you set $channel instead of calling skip() in lexer rules.)
See Also
org.antlr.runtime.UnbufferedTokenStream, org.antlr.runtime.BufferedTokenStream

Field Summary

Modifier and TypeField and Description
protected int
channel

Skip tokens on any channel but this one; this is how we skip whitespace...

Inherited from org.antlr.runtime.BufferedTokenStream:
lastMarkerprangetokenstokenSource

Constructor Summary

AccessConstructor and Description
public
public
public
CommonTokenStream(TokenSource tokenSource, int channel)

Method Summary

Modifier and TypeMethod and Description
public void
consume()

Overrides org.antlr.runtime.BufferedTokenStream.consume.

Implements org.antlr.runtime.IntStream.consume.

Always leave p on an on-channel token.
public int
getNumberOfOnChannelTokens()

Count EOF just once.

protected Token
public Token
LT(int k)

Overrides org.antlr.runtime.BufferedTokenStream.LT.

Implements org.antlr.runtime.TokenStream.LT.

Get Token at current input pointer + i ahead where i=1 is next Token.
public void
public void
setTokenSource(TokenSource tokenSource)

Overrides org.antlr.runtime.BufferedTokenStream.setTokenSource.

Reset this token stream by setting its token source.
protected void
protected int
skipOffTokenChannels(int i)

Given a starting index, return the index of the first on-channel token.

protected int
Inherited from org.antlr.runtime.BufferedTokenStream:
fetchfillgetgetgetSourceNamegetTokensgetTokensgetTokensgetTokensgetTokensgetTokenSourceindexLAmarkrangereleaserewindrewindseeksizesynctoStringtoStringtoString

Field Detail

channelback to summary
protected int channel

Skip tokens on any channel but this one; this is how we skip whitespace...

Constructor Detail

CommonTokenStreamback to summary
public CommonTokenStream()
CommonTokenStreamback to summary
public CommonTokenStream(TokenSource tokenSource)
CommonTokenStreamback to summary
public CommonTokenStream(TokenSource tokenSource, int channel)

Method Detail

consumeback to summary
public void consume()

Overrides org.antlr.runtime.BufferedTokenStream.consume.

Implements org.antlr.runtime.IntStream.consume.

Always leave p on an on-channel token.

Annotations
@Override
getNumberOfOnChannelTokensback to summary
public int getNumberOfOnChannelTokens()

Count EOF just once.

LBback to summary
protected Token LB(int k)

Overrides org.antlr.runtime.BufferedTokenStream.LB.

Annotations
@Override
LTback to summary
public Token LT(int k)

Overrides org.antlr.runtime.BufferedTokenStream.LT.

Implements org.antlr.runtime.TokenStream.LT.

Doc from org.antlr.runtime.TokenStream.LT.

Get Token at current input pointer + i ahead where i=1 is next Token. i<0 indicates tokens in the past. So -1 is previous token and -2 is two tokens ago. LT(0) is undefined. For i>=n, return Token.EOFToken. Return null for LT(0) and any index that results in an absolute address that is negative.

Annotations
@Override
resetback to summary
public void reset()

Overrides org.antlr.runtime.BufferedTokenStream.reset.

Annotations
@Override
setTokenSourceback to summary
public void setTokenSource(TokenSource tokenSource)

Overrides org.antlr.runtime.BufferedTokenStream.setTokenSource.

Reset this token stream by setting its token source.

Annotations
@Override
setupback to summary
protected void setup()

Overrides org.antlr.runtime.BufferedTokenStream.setup.

Annotations
@Override
skipOffTokenChannelsback to summary
protected int skipOffTokenChannels(int i)

Given a starting index, return the index of the first on-channel token.

skipOffTokenChannelsReverseback to summary
protected int skipOffTokenChannelsReverse(int i)