Top Description Fields Constructors Methods
org.antlr.v4.runtime

public Class UnbufferedCharStream

extends Object
implements CharStream
Class Inheritance
All Implemented Interfaces
org.antlr.v4.runtime.CharStream, org.antlr.v4.runtime.IntStream
Imports
org.antlr.v4.runtime.misc.Interval, java.io.IOException, .InputStream, .InputStreamReader, .Reader, java.nio.charset.Charset, .StandardCharsets, java.util.Arrays
Do not buffer up the entire char stream. It does keep a small buffer for efficiency and also buffers while a mark exists (set by the lookahead prediction in parser). "Unbuffered" here refers to fact that it doesn't buffer all data, not that's it's on demand loading of char. Before 4.7, this class used the default environment encoding to convert bytes to UTF-16, and held the UTF-16 bytes in the buffer as chars. As of 4.7, the class uses UTF-8 by default, and the buffer holds Unicode code points in the buffer as ints.

Field Summary

Modifier and TypeField and Description
protected int
currentCharIndex

Absolute character index.

protected int[]
data

A moving window buffer of the data being scanned.

protected Reader
input

protected int
lastChar

This is the LA(-1) character for the current position.

protected int
lastCharBufferStart

When numMarkers > 0, this is the LA(-1) character for the first character in data.

protected int
n

The number of characters currently in data.

public String
name

The name or source of this char stream.

protected int
numMarkers

Count up with mark() and down with release().

protected int
p

0..n-1 index into data of next character.

Constructor Summary

AccessConstructor and Description
public
UnbufferedCharStream()

Useful for subclasses that pull char from other than this.input.

public
UnbufferedCharStream(int bufferSize)

Useful for subclasses that pull char from other than this.input.

public
UnbufferedCharStream(InputStream input)

public
UnbufferedCharStream(Reader input)

public
UnbufferedCharStream(InputStream input, int bufferSize)

public
UnbufferedCharStream(InputStream input, int bufferSize, Charset charset)

public
UnbufferedCharStream(Reader input, int bufferSize)

Method Summary

Modifier and TypeMethod and Description
protected void
add(int c)

public void
consume()

Implements org.antlr.v4.runtime.IntStream.consume.

Consumes the current symbol in the stream.
protected int
fill(int n)

Add n characters to the buffer.

protected final int
getBufferStartIndex()

public String
getSourceName()

Implements org.antlr.v4.runtime.IntStream.getSourceName.

Gets the name of the underlying symbol source.
public String
getText(Interval
an interval within the stream
interval)

Implements org.antlr.v4.runtime.CharStream.getText.

This method returns the text for a range of characters within this input stream.
public int
index()

Implements org.antlr.v4.runtime.IntStream.index.

Return the index into the stream of the input symbol referred to by LA(1).
public int
LA(int i)

Implements org.antlr.v4.runtime.IntStream.LA.

Gets the value of the symbol at offset i from the current position.
public int
mark()

Implements org.antlr.v4.runtime.IntStream.mark.

Return a marker that we can release later.
protected int
nextChar()

Override to provide different source of characters than input.

public void
release(int marker)

Implements org.antlr.v4.runtime.IntStream.release.

Decrement number of markers, resetting buffer if we hit 0.
public void
seek(int
The absolute index to seek to.
index)

Implements org.antlr.v4.runtime.IntStream.seek.

Seek to absolute character index, which might not be in the current sliding window.
public int
size()

Implements org.antlr.v4.runtime.IntStream.size.

Returns the total number of symbols in the stream, including a single EOF symbol.
protected void
sync(int want)

Make sure we have 'need' elements from current position p.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

currentCharIndexback to summary
protected int currentCharIndex

Absolute character index. It's the index of the character about to be read via LA(1). Goes from 0 to the number of characters in the entire stream, although the stream size is unknown before the end is reached.

databack to summary
protected int[] data

A moving window buffer of the data being scanned. While there's a marker, we keep adding to buffer. Otherwise, consume() resets so we start filling at index 0 again.

inputback to summary
protected Reader input
lastCharback to summary
protected int lastChar

This is the LA(-1) character for the current position.

lastCharBufferStartback to summary
protected int lastCharBufferStart

When numMarkers > 0, this is the LA(-1) character for the first character in data. Otherwise, this is unspecified.

nback to summary
protected int n

The number of characters currently in data.

This is not the buffer capacity, that's data.length.

nameback to summary
public String name

The name or source of this char stream.

numMarkersback to summary
protected int numMarkers

Count up with mark() and down with release(). When we release() the last mark, numMarkers reaches 0 and we reset the buffer. Copy data[p]..data[n-1] to data[0]..data[(n-1)-p].

pback to summary
protected int p

0..n-1 index into data of next character.

The LA(1) character is data[p]. If p == n, we are out of buffered characters.

Constructor Detail

UnbufferedCharStreamback to summary
public UnbufferedCharStream()

Useful for subclasses that pull char from other than this.input.

UnbufferedCharStreamback to summary
public UnbufferedCharStream(int bufferSize)

Useful for subclasses that pull char from other than this.input.

UnbufferedCharStreamback to summary
public UnbufferedCharStream(InputStream input)
UnbufferedCharStreamback to summary
public UnbufferedCharStream(Reader input)
UnbufferedCharStreamback to summary
public UnbufferedCharStream(InputStream input, int bufferSize)
UnbufferedCharStreamback to summary
public UnbufferedCharStream(InputStream input, int bufferSize, Charset charset)
UnbufferedCharStreamback to summary
public UnbufferedCharStream(Reader input, int bufferSize)

Method Detail

addback to summary
protected void add(int c)
consumeback to summary
public void consume()

Implements org.antlr.v4.runtime.IntStream.consume.

Doc from org.antlr.v4.runtime.IntStream.consume.

Consumes the current symbol in the stream. This method has the following effects:

  • Forward movement: The value of index() before calling this method is less than the value of index() after calling this method.
  • Ordered lookahead: The value of LA(1) before calling this method becomes the value of LA(-1) after calling this method.
Note that calling this method does not guarantee that index() is incremented by exactly 1, as that would preclude the ability to implement filtering streams (e.g. CommonTokenStream which distinguishes between "on-channel" and "off-channel" tokens).
Annotations
@Override
fillback to summary
protected int fill(int n)

Add n characters to the buffer. Returns the number of characters actually added to the buffer. If the return value is less than n, then EOF was reached before n characters could be added.

getBufferStartIndexback to summary
protected final int getBufferStartIndex()
getSourceNameback to summary
public String getSourceName()

Implements org.antlr.v4.runtime.IntStream.getSourceName.

Doc from org.antlr.v4.runtime.IntStream.getSourceName.

Gets the name of the underlying symbol source. This method returns a non-null, non-empty string. If such a name is not known, this method returns UNKNOWN_SOURCE_NAME.

Annotations
@Override
getTextback to summary
public String getText(Interval interval)

Implements org.antlr.v4.runtime.CharStream.getText.

Doc from org.antlr.v4.runtime.CharStream.getText.

This method returns the text for a range of characters within this input stream. This method is guaranteed to not throw an exception if the specified interval lies entirely within a marked range. For more information about marked ranges, see IntStream#mark.

Parameters
interval:Interval

an interval within the stream

Returns:String

the text of the specified interval

Annotations
@Override
indexback to summary
public int index()

Implements org.antlr.v4.runtime.IntStream.index.

Doc from org.antlr.v4.runtime.IntStream.index.

Return the index into the stream of the input symbol referred to by LA(1).

The behavior of this method is unspecified if no call to an initializing method has occurred after this stream was constructed.

Annotations
@Override
LAback to summary
public int LA(int i)

Implements org.antlr.v4.runtime.IntStream.LA.

Doc from org.antlr.v4.runtime.IntStream.LA.

Gets the value of the symbol at offset i from the current position. When i==1, this method returns the value of the current symbol in the stream (which is the next symbol to be consumed). When i==-1, this method returns the value of the previously read symbol in the stream. It is not valid to call this method with i==0, but the specific behavior is unspecified because this method is frequently called from performance-critical code.

This method is guaranteed to succeed if any of the following are true:

  • i>0
  • i==-1 and index() returns a value greater than the value of index() after the stream was constructed and LA(1) was called in that order. Specifying the current index() relative to the index after the stream was created allows for filtering implementations that do not return every symbol from the underlying source. Specifying the call to LA(1) allows for lazily initialized streams.
  • LA(i) refers to a symbol consumed within a marked region that has not yet been released.

If i represents a position at or beyond the end of the stream, this method returns EOF.

The return value is unspecified if i<0 and fewer than -i calls to consume() have occurred from the beginning of the stream before calling this method.

Annotations
@Override
markback to summary
public int mark()

Implements org.antlr.v4.runtime.IntStream.mark.

Return a marker that we can release later.

The specific marker value used for this class allows for some level of protection against misuse where seek() is called on a mark or release() is called in the wrong order.

Returns:int

Doc from org.antlr.v4.runtime.IntStream.mark.

An opaque marker which should be passed to release() when the marked range is no longer required.

Annotations
@Override
nextCharback to summary
protected int nextChar() throws IOException

Override to provide different source of characters than input.

releaseback to summary
public void release(int marker)

Implements org.antlr.v4.runtime.IntStream.release.

Decrement number of markers, resetting buffer if we hit 0.

Annotations
@Override
seekback to summary
public void seek(int index)

Implements org.antlr.v4.runtime.IntStream.seek.

Seek to absolute character index, which might not be in the current sliding window. Move p to index-bufferStartIndex.

Parameters
index:int

Doc from org.antlr.v4.runtime.IntStream.seek.

The absolute index to seek to.

Annotations
@Override
sizeback to summary
public int size()

Implements org.antlr.v4.runtime.IntStream.size.

Doc from org.antlr.v4.runtime.IntStream.size.

Returns the total number of symbols in the stream, including a single EOF symbol.

Annotations
@Override
syncback to summary
protected void sync(int want)

Make sure we have 'need' elements from current position p. Last valid p index is data.length-1. p+need-1 is the char index 'need' elements ahead. If we need 1 element, (p+1-1)==p must be less than data.length.