Top Description Fields Constructors Methods
org.codehaus.janino

public Class TokenStreamImpl

extends Object
implements TokenStream
Class Inheritance
All Implemented Interfaces
org.codehaus.janino.TokenStream
Imports
java.io.IOException, org.codehaus.commons.compiler.CompileException, .Location, .WarningHandler, org.codehaus.commons.nullanalysis.Nullable
Standard implementation for the TokenStream.

Field Summary

Modifier and TypeField and Description
private String
docComment

The optional JAVADOC comment preceding the nextToken.

private Token
nextButOneToken

private Token
nextToken

private Token
previousToken

private final Scanner
scanner

private WarningHandler
warningHandler

Constructor Summary

AccessConstructor and Description
public
TokenStreamImpl(Scanner scanner)

Method Summary

Modifier and TypeMethod and Description
protected final CompileException
compileException(String message)

Convenience method for throwing a CompileException.

public String

Returns:

null if the next token is not preceded by a doc comment
doc()

Gets the text of the doc comment (a.k.a. "JAVADOC comment") preceding the next token.

private static int
indexOf(String[] strings, String subject)

private static int
indexOf(TokenType[] tta, TokenType subject)

private static String
join(Object[] oa, String glue)

public Location
location()

Implements org.codehaus.janino.TokenStream.location.

public Token
peek()

Implements org.codehaus.janino.TokenStream.peek.

public boolean
peek(String suspected)

Implements org.codehaus.janino.TokenStream.peek.

public int
peek(String... suspected)

Implements org.codehaus.janino.TokenStream.peek.

Checks whether the value of the next token equals any of the suspected; does not consume the next token.
public boolean
peek(TokenType suspected)

Implements org.codehaus.janino.TokenStream.peek.

public int
peek(TokenType... suspected)

Implements org.codehaus.janino.TokenStream.peek.

Checks whether the type of the next token is any of the suspected; does not consume the next token.
public Token
peekNextButOne()

Implements org.codehaus.janino.TokenStream.peekNextButOne.

public boolean
peekNextButOne(String suspected)

Implements org.codehaus.janino.TokenStream.peekNextButOne.

public boolean
peekRead(String suspected)

Implements org.codehaus.janino.TokenStream.peekRead.

Checks whether the value of the next token equals the suspected; if so, consumes the token.
public int
peekRead(String... suspected)

Implements org.codehaus.janino.TokenStream.peekRead.

Checks whether the value of the next token is one of the suspected; if so, consumes the token.
public String
peekRead(TokenType suspected)

Implements org.codehaus.janino.TokenStream.peekRead.

Checks whether the type of the next token is the suspected; if so, consumes the token.
public int
peekRead(TokenType... suspected)

Implements org.codehaus.janino.TokenStream.peekRead.

Checks whether the type of the next token is one of the suspected; if so, consumes the token.
private Token
produceToken()

public Token
read()

Implements org.codehaus.janino.TokenStream.read.

public void
read(String expected)

Implements org.codehaus.janino.TokenStream.read.

Verifies that the value of the next token equals expected, and consumes the token.
public int
read(String... expected)

Implements org.codehaus.janino.TokenStream.read.

Verifies that the value of the next token equals one of the expected, and consumes the token.
public String
read(TokenType expected)

Implements org.codehaus.janino.TokenStream.read.

Verifies that the type of the next token is the expected, and consumes the token.
public int
read(TokenType... expected)

Implements org.codehaus.janino.TokenStream.read.

Verifies that the type of the next token is one of the expected, and consumes the token.
public void
setWarningHandler(WarningHandler
null to indicate that no warnings be issued
warningHandler)

Implements org.codehaus.janino.TokenStream.setWarningHandler.

By default, warnings are discarded, but an application my install a WarningHandler.
public String
toString()

Overrides java.lang.Object.toString.

Returns a string representation of the object.
private void
warning(String handle, String message, Location location)

Issues a warning with the given message and location and returns.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAllwaitwaitwait

Field Detail

docCommentback to summary
private String docComment

The optional JAVADOC comment preceding the nextToken.

Annotations
@Nullable
nextButOneTokenback to summary
private Token nextButOneToken
Annotations
@Nullable
nextTokenback to summary
private Token nextToken
Annotations
@Nullable
previousTokenback to summary
private Token previousToken
Annotations
@Nullable
scannerback to summary
private final Scanner scanner
warningHandlerback to summary
private WarningHandler warningHandler
Annotations
@Nullable

Constructor Detail

TokenStreamImplback to summary
public TokenStreamImpl(Scanner scanner)

Method Detail

compileExceptionback to summary
protected final CompileException compileException(String message)

Convenience method for throwing a CompileException.

docback to summary
public String doc()

Gets the text of the doc comment (a.k.a. "JAVADOC comment") preceding the next token.

Returns:String

null if the next token is not preceded by a doc comment

Annotations
@Nullable
indexOfback to summary
private static int indexOf(String[] strings, String subject)
indexOfback to summary
private static int indexOf(TokenType[] tta, TokenType subject)
joinback to summary
private static String join(Object[] oa, String glue)
locationback to summary
public Location location()

Implements org.codehaus.janino.TokenStream.location.

Returns:Location

Doc from org.codehaus.janino.TokenStream.location.

The location of the first character of the previously read (not peeked!) token

Annotations
@Override
peekback to summary
public Token peek() throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peek.

Returns:Token

Doc from org.codehaus.janino.TokenStream.peek.

The next token, but does not consume it

Annotations
@Override
peekback to summary
public boolean peek(String suspected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peek.

Returns:boolean

Doc from org.codehaus.janino.TokenStream.peek.

Whether the value of the next token equals suspected; does not consume the next token

Annotations
@Override
peekback to summary
public int peek(String... suspected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peek.

Doc from org.codehaus.janino.TokenStream.peek.

Checks whether the value of the next token equals any of the suspected; does not consume the next token.

Returns:int

The index of the first of the suspected that equals the value of the next token, or -1 if the value of the next token equals none of the suspected

Annotations
@Override
peekback to summary
public boolean peek(TokenType suspected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peek.

Returns:boolean

Doc from org.codehaus.janino.TokenStream.peek.

Whether the type of the next token is suspected

Annotations
@Override
peekback to summary
public int peek(TokenType... suspected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peek.

Doc from org.codehaus.janino.TokenStream.peek.

Checks whether the type of the next token is any of the suspected; does not consume the next token.

Returns:int

The index of the first of the suspected types that is the next token's type, or -1 if the type of the next token is none of the suspected types

Annotations
@Override
peekNextButOneback to summary
public Token peekNextButOne() throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peekNextButOne.

Returns:Token

Doc from org.codehaus.janino.TokenStream.peekNextButOne.

The next-but-one token, but consumes neither the next nor the next-but-one token

Annotations
@Override
peekNextButOneback to summary
public boolean peekNextButOne(String suspected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peekNextButOne.

Returns:boolean

Doc from org.codehaus.janino.TokenStream.peekNextButOne.

Whether the value of the next-but-one token equals the suspected; consumes neither the next nor the next-but-one token

Annotations
@Override
peekReadback to summary
public boolean peekRead(String suspected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peekRead.

Doc from org.codehaus.janino.TokenStream.peekRead.

Checks whether the value of the next token equals the suspected; if so, consumes the token.

Annotations
@Override
peekReadback to summary
public int peekRead(String... suspected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peekRead.

Doc from org.codehaus.janino.TokenStream.peekRead.

Checks whether the value of the next token is one of the suspected; if so, consumes the token.

Returns:int

The index of the first of the suspected that equals the next token's value; -1 iff the next token's value equals none of the suspected

Annotations
@Override
peekReadback to summary
public String peekRead(TokenType suspected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peekRead.

Doc from org.codehaus.janino.TokenStream.peekRead.

Checks whether the type of the next token is the suspected; if so, consumes the token.

Returns:String

The value of the next token; an interned String iff the token represents an identifier, true, false, null, or an operator

Annotations
@Override
@Nullable
peekReadback to summary
public int peekRead(TokenType... suspected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.peekRead.

Doc from org.codehaus.janino.TokenStream.peekRead.

Checks whether the type of the next token is one of the suspected; if so, consumes the token.

Returns:int

The index of the elements that is the next token's type, or -1 iff the type of next token is none of the suspected

Annotations
@Override
produceTokenback to summary
private Token produceToken() throws CompileException, IOException
readback to summary
public Token read() throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.read.

Returns:Token

Doc from org.codehaus.janino.TokenStream.read.

The next token, which it also consumes, or null iff the scanner is at end-of-input

Annotations
@Override
readback to summary
public void read(String expected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.read.

Doc from org.codehaus.janino.TokenStream.read.

Verifies that the value of the next token equals expected, and consumes the token.

Annotations
@Override
Exceptions
CompileException:
The value of the next token does not equal expected (this includes the case that the scanner is at end-of-input)
readback to summary
public int read(String... expected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.read.

Doc from org.codehaus.janino.TokenStream.read.

Verifies that the value of the next token equals one of the expected, and consumes the token.

Returns:int

The index of the consumed token within expected

Annotations
@Override
Exceptions
CompileException:
The value of the next token does not equal any of the expected (this includes the case where the scanner is at end-of-input)
readback to summary
public String read(TokenType expected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.read.

Doc from org.codehaus.janino.TokenStream.read.

Verifies that the type of the next token is the expected, and consumes the token.

Returns:String

The value of the next token; an interned String iff the token represents an identifier, true, false, null, or an operator

Annotations
@Override
Exceptions
CompileException:
The next token's type is not the expected
readback to summary
public int read(TokenType... expected) throws CompileException, IOException

Implements org.codehaus.janino.TokenStream.read.

Doc from org.codehaus.janino.TokenStream.read.

Verifies that the type of the next token is one of the expected, and consumes the token.

Returns:int

The index of the first of the expected types that is the next token's type; -1 if the type of the next token is none of the expected

Annotations
@Override
Exceptions
CompileException:
The next token's type is none of the expected
setWarningHandlerback to summary
public void setWarningHandler(WarningHandler warningHandler)

Implements org.codehaus.janino.TokenStream.setWarningHandler.

Doc from org.codehaus.janino.TokenStream.setWarningHandler.

By default, warnings are discarded, but an application my install a WarningHandler.

Notice that there is no Parser.setErrorHandler() method, but parse errors always throw a CompileException. The reason being is that there is no reasonable way to recover from parse errors and continue parsing, so there is no need to install a custom parse error handler.

Parameters
warningHandler:WarningHandler

null to indicate that no warnings be issued

Annotations
@Override
toStringback to summary
public String toString()

Overrides java.lang.Object.toString.

Doc from java.lang.Object.toString.

Returns a string representation of the object.

Returns:String

a string representation of the object.

Annotations
@Override
warningback to summary
private void warning(String handle, String message, Location location) throws CompileException

Issues a warning with the given message and location and returns. This is done through a WarningHandler that was installed through setWarningHandler(WarningHandler).

The handle argument qualifies the warning and is typically used by the WarningHandler to suppress individual warnings.

Exceptions
CompileException:
The optionally installed WarningHandler decided to throw a CompileException