Top Description Fields Constructors Methods
com.ibm.icu.impl.data

public Class TokenIterator

extends Object
Class Inheritance
Imports
java.io.IOException, com.ibm.icu.impl.PatternProps, .Utility, com.ibm.icu.text.UTF16
An iterator class that returns successive string tokens from some source. String tokens are, in general, separated by Pattern_White_Space in the source test. Furthermore, they may be delimited by either single or double quotes (opening and closing quotes must match). Escapes are processed using standard ICU unescaping.

2015-sep-03 TODO: Only used in com.ibm.icu.dev.test.format, move there.

Field Summary

Modifier and TypeField and Description
private StringBuffer
buf

private boolean
done

private int
lastpos

private String
line

private int
pos

private ResourceReader
reader

Constructor Summary

AccessConstructor and Description
public
TokenIterator(ResourceReader r)

Construct an iterator over the tokens returned by the given ResourceReader, ignoring blank lines and comment lines (first non-blank character is '#').

Method Summary

Modifier and TypeMethod and Description
public String
describePosition()

Return a string description of the position of the last line returned by readLine() or readLineSkippingComments().

public int
getLineNumber()

Return the one-based line number of the line of the last token returned by next().

public String
next()

Return the next token from this iterator, or null if the last token has been returned.

private int

Returns:

offset to the next character to read from line, or if the end of the line is reached without scanning a valid token, -1
nextToken(int
the offset into the string
position)

Read the next token from 'this.line' and append it to 'this.buf'.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

bufback to summary
private StringBuffer buf
doneback to summary
private boolean done
lastposback to summary
private int lastpos
lineback to summary
private String line
posback to summary
private int pos
readerback to summary
private ResourceReader reader

Constructor Detail

TokenIteratorback to summary
public TokenIterator(ResourceReader r)

Construct an iterator over the tokens returned by the given ResourceReader, ignoring blank lines and comment lines (first non-blank character is '#'). Note that trailing comments on a line, beginning with the first unquoted '#', are recognized.

Method Detail

describePositionback to summary
public String describePosition()

Return a string description of the position of the last line returned by readLine() or readLineSkippingComments().

getLineNumberback to summary
public int getLineNumber()

Return the one-based line number of the line of the last token returned by next(). Should only be called after a call to next(); otherwise the return value is undefined.

nextback to summary
public String next() throws IOException

Return the next token from this iterator, or null if the last token has been returned.

nextTokenback to summary
private int nextToken(int position)

Read the next token from 'this.line' and append it to 'this.buf'. Tokens are separated by Pattern_White_Space. Tokens may also be delimited by double or single quotes. The closing quote must match the opening quote. If a '#' is encountered, the rest of the line is ignored, unless it is backslash-escaped or within quotes.

Parameters
position:int

the offset into the string

Returns:int

offset to the next character to read from line, or if the end of the line is reached without scanning a valid token, -1