Apache Commons CSV 1.14.0 · org.apache.commons.csv.Lexer

Modifier and Type	Field and Description
private final int	commentStart
private static final String	CR_STRING
private final char[]	delimiter
private final char[]	delimiterBuf
private final int	escape
private final char[]	escapeDelimiterBuf
private String	firstEol
private final boolean	ignoreEmptyLines
private final boolean	ignoreSurroundingSpaces
private boolean	isLastTokenDelimiter
private final boolean	lenientEof
private static final String	LF_STRING
private final int	quoteChar
private final ExtendedBufferedReader	reader The buffered reader.
private final boolean	trailingData

Access	Constructor and Description
pack-priv	Lexer(final CSVFormat format, final ExtendedBufferedReader reader)

Method Summary

Modifier and Type	Method and Description
private void	appendNextEscapedCharacterToToken(final Token the current token token) Appends the next escaped character to the token's content.
public void	close() Implements java.io.Closeable.close. Closes resources.
pack-priv long	Returns: the number of bytes read getBytesRead() Gets the number of bytes read
pack-priv long	Returns: the current character position getCharacterPosition() Returns the current character position
pack-priv long	Returns: the current line number getCurrentLineNumber() Returns the current line number
pack-priv String	getFirstEol()
pack-priv boolean	isClosed()
pack-priv boolean	isCommentStart(final int ch)
pack-priv boolean	Returns: true if the next characters constitute a delimiter. isDelimiter(final int the current character. ch) Determine whether the next characters constitute a delimiter through `ExtendedBufferedReader#peek(char[])`.
pack-priv boolean	Returns: true if the given character indicates the end of the file. isEndOfFile(final int ch) Tests if the given character indicates the end of the file.
pack-priv boolean	Returns: true if the given character is the escape character. isEscape(final int ch) Tests if the given character is the escape character.
pack-priv boolean	Returns: true if the next characters constitute an escape delimiter. isEscapeDelimiter() Tests if the next characters constitute a escape delimiter through `ExtendedBufferedReader#peek(char[])`.
private boolean	isMetaChar(final int ch)
pack-priv boolean	isQuoteChar(final int ch)
pack-priv boolean	Returns: true if the character is at the start of a line. isStartOfLine(final int the character to check ch) Tests if the current character represents the start of a line: a CR, LF, or is at the start of the file.
pack-priv Token	Returns: the next token found. nextToken(final Token an existing Token object to reuse. The caller is responsible for initializing the Token. token) Returns the next token.
private int	nullToDisabled(final Character c)
private Token	Returns: a valid token object parseEncapsulatedToken(final Token the current token token) Parses an encapsulated token.
private Token	Returns: the filled token parseSimpleToken(final Token the current token token, final int the current character ch) Parses a simple token.
pack-priv boolean	Returns: true if the given or next character is a line-terminator readEndOfLine(final int ch) Greedily accepts \n, \r and \r\n This checker consumes silently the second control-character...
pack-priv int	Returns: the unescaped character (as an int) or `IOUtils#EOF` if char following the escape is invalid. readEscape() Handle an escape sequence.
pack-priv void	trimTrailingSpaces(final StringBuilder buffer)

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

commentStart	back to summary
private final int commentStart

CR_STRING	back to summary
private static final String CR_STRING

delimiter	back to summary
private final char[] delimiter

delimiterBuf	back to summary
private final char[] delimiterBuf

escape	back to summary
private final int escape

escapeDelimiterBuf	back to summary
private final char[] escapeDelimiterBuf

firstEol	back to summary
private String firstEol

ignoreEmptyLines	back to summary
private final boolean ignoreEmptyLines

ignoreSurroundingSpaces	back to summary
private final boolean ignoreSurroundingSpaces

isLastTokenDelimiter	back to summary
private boolean isLastTokenDelimiter

lenientEof	back to summary
private final boolean lenientEof

LF_STRING	back to summary
private static final String LF_STRING

quoteChar	back to summary
private final int quoteChar

reader	back to summary
private final ExtendedBufferedReader reader The buffered reader.

trailingData	back to summary
private final boolean trailingData

Lexer	back to summary
pack-priv Lexer(final CSVFormat format, final ExtendedBufferedReader reader)

Method Detail

appendNextEscapedCharacterToToken	back to summary
private void appendNextEscapedCharacterToToken(final Token token) throws IOException Appends the next escaped character to the token's content. Parameters token:Token the current token Exceptions IOException: on stream access error CSVException: Thrown on invalid input.

appendNextEscapedCharacterToToken

back to summary

private void appendNextEscapedCharacterToToken(final Token token) throws IOException

Appends the next escaped character to the token's content.

Parameters

token:Token: the current token

Exceptions

IOException:: on stream access error
CSVException:: Thrown on invalid input.

close	back to summary
public void close() throws IOException Implements java.io.Closeable.close. Closes resources. Annotations @Override Exceptions IOException: If an I/O error occurs

back to summary

public void close() throws IOException

Implements java.io.Closeable.close.

Closes resources.

Annotations

@Override

Exceptions

IOException:: If an I/O error occurs

getBytesRead	back to summary
pack-priv long getBytesRead() Gets the number of bytes read Returns:long the number of bytes read

getBytesRead

back to summary

pack-priv long getBytesRead()

Gets the number of bytes read

Returns:long: the number of bytes read

getCharacterPosition	back to summary
pack-priv long getCharacterPosition() Returns the current character position Returns:long the current character position

getCharacterPosition

back to summary

pack-priv long getCharacterPosition()

Returns the current character position

Returns:long: the current character position

getCurrentLineNumber	back to summary
pack-priv long getCurrentLineNumber() Returns the current line number Returns:long the current line number

getCurrentLineNumber

back to summary

pack-priv long getCurrentLineNumber()

Returns the current line number

Returns:long: the current line number

getFirstEol	back to summary
pack-priv String getFirstEol()

isClosed	back to summary
pack-priv boolean isClosed()

isCommentStart	back to summary
pack-priv boolean isCommentStart(final int ch)

isDelimiter back to summary

isDelimiter	back to summary
pack-priv boolean isDelimiter(final int ch) throws IOException Determine whether the next characters constitute a delimiter through `ExtendedBufferedReader#peek(char[])`. Parameters ch:int the current character. Returns:boolean true if the next characters constitute a delimiter. Exceptions IOException: If an I/O error occurs.

pack-priv boolean isDelimiter(final int ch) throws IOException

Determine whether the next characters constitute a delimiter through ExtendedBufferedReader#peek(char[]).

Parameters

ch:int: the current character.

Returns:boolean

true if the next characters constitute a delimiter.

Exceptions

IOException:: If an I/O error occurs.

isEndOfFile	back to summary
pack-priv boolean isEndOfFile(final int ch) Tests if the given character indicates the end of the file. Returns:boolean true if the given character indicates the end of the file.

isEndOfFile

back to summary

pack-priv boolean isEndOfFile(final int ch)

Tests if the given character indicates the end of the file.

Returns:boolean: true if the given character indicates the end of the file.

isEscape	back to summary
pack-priv boolean isEscape(final int ch) Tests if the given character is the escape character. Returns:boolean true if the given character is the escape character.

isEscape

back to summary

pack-priv boolean isEscape(final int ch)

Tests if the given character is the escape character.

Returns:boolean: true if the given character is the escape character.

isEscapeDelimiter back to summary

isEscapeDelimiter	back to summary
pack-priv boolean isEscapeDelimiter() throws IOException Tests if the next characters constitute a escape delimiter through `ExtendedBufferedReader#peek(char[])`. For example, for delimiter "[\|]" and escape '!', return true if the next characters constitute "![!\|!]". Returns:boolean true if the next characters constitute an escape delimiter. Exceptions IOException: If an I/O error occurs.

pack-priv boolean isEscapeDelimiter() throws IOException

Tests if the next characters constitute a escape delimiter through ExtendedBufferedReader#peek(char[]). For example, for delimiter "[|]" and escape '!', return true if the next characters constitute "![!|!]".

Returns:boolean

true if the next characters constitute an escape delimiter.

Exceptions

IOException:: If an I/O error occurs.

isMetaChar	back to summary
private boolean isMetaChar(final int ch)

isQuoteChar	back to summary
pack-priv boolean isQuoteChar(final int ch)

isStartOfLine	back to summary
pack-priv boolean isStartOfLine(final int ch) Tests if the current character represents the start of a line: a CR, LF, or is at the start of the file. Parameters ch:int the character to check Returns:boolean true if the character is at the start of a line.

isStartOfLine

back to summary

pack-priv boolean isStartOfLine(final int ch)

Tests if the current character represents the start of a line: a CR, LF, or is at the start of the file.

Parameters

ch:int: the character to check

Returns:boolean

true if the character is at the start of a line.

nextToken	back to summary
pack-priv Token nextToken(final Token token) throws IOException Returns the next token. A token corresponds to a term, a record change or an end-of-file indicator. Parameters token:Token an existing Token object to reuse. The caller is responsible for initializing the Token. Returns:Token the next token found. Exceptions IOException: on stream access error. CSVException: Thrown on invalid input.

nextToken

back to summary

pack-priv Token nextToken(final Token token) throws IOException

Returns the next token.

A token corresponds to a term, a record change or an end-of-file indicator.

Parameters

token:Token: an existing Token object to reuse. The caller is responsible for initializing the Token.

Returns:Token

the next token found.

Exceptions

IOException:: on stream access error.
CSVException:: Thrown on invalid input.

nullToDisabled	back to summary
private int nullToDisabled(final Character c)

parseEncapsulatedToken	back to summary
private Token parseEncapsulatedToken(final Token token) throws IOException Parses an encapsulated token. Encapsulated tokens are surrounded by the given encapsulating string. The encapsulator itself might be included in the token using a doubling syntax (as "", '') or using escaping (as in \", \'). Whitespaces before and after an encapsulated token is ignored. The token is finished when one of the following conditions becomes true: An unescaped encapsulator has been reached and is followed by optional whitespace then: delimiter (TOKEN) end of line (EORECORD) end of stream has been reached (EOF) Parameters token:Token the current token Returns:Token a valid token object Exceptions IOException: Thrown when in an invalid state: EOF before closing encapsulator or invalid character before delimiter or EOL. CSVException: Thrown on invalid input.

parseEncapsulatedToken

back to summary

private Token parseEncapsulatedToken(final Token token) throws IOException

Parses an encapsulated token.

Encapsulated tokens are surrounded by the given encapsulating string. The encapsulator itself might be included in the token using a doubling syntax (as "", '') or using escaping (as in \", \'). Whitespaces before and after an encapsulated token is ignored. The token is finished when one of the following conditions becomes true:

An unescaped encapsulator has been reached and is followed by optional whitespace then:

delimiter (TOKEN)
end of line (EORECORD)

end of stream has been reached (EOF)

Parameters

token:Token: the current token

Returns:Token

a valid token object

Exceptions

IOException:: Thrown when in an invalid state: EOF before closing encapsulator or invalid character before delimiter or EOL.
CSVException:: Thrown on invalid input.

parseSimpleToken	back to summary
private Token parseSimpleToken(final Token token, final int ch) throws IOException Parses a simple token. Simple tokens are tokens that are not surrounded by encapsulators. A simple token might contain escaped delimiters (as \, or \;). The token is finished when one of the following conditions becomes true: The end of line has been reached (EORECORD) The end of stream has been reached (EOF) An unescaped delimiter has been reached (TOKEN) Parameters token:Token the current token ch:int the current character Returns:Token the filled token Exceptions IOException: on stream access error CSVException: Thrown on invalid input.

parseSimpleToken

back to summary

private Token parseSimpleToken(final Token token, final int ch) throws IOException

Parses a simple token.

Simple tokens are tokens that are not surrounded by encapsulators. A simple token might contain escaped delimiters (as \, or \;). The token is finished when one of the following conditions becomes true:

The end of line has been reached (EORECORD)
The end of stream has been reached (EOF)
An unescaped delimiter has been reached (TOKEN)

Parameters

token:Token: the current token
ch:int: the current character

Returns:Token

the filled token

Exceptions

IOException:: on stream access error
CSVException:: Thrown on invalid input.

readEndOfLine	back to summary
pack-priv boolean readEndOfLine(final int ch) throws IOException Greedily accepts \n, \r and \r\n This checker consumes silently the second control-character... Returns:boolean true if the given or next character is a line-terminator

readEndOfLine

back to summary

pack-priv boolean readEndOfLine(final int ch) throws IOException

Greedily accepts \n, \r and \r\n This checker consumes silently the second control-character...

Returns:boolean: true if the given or next character is a line-terminator

readEscape back to summary

readEscape	back to summary
pack-priv int readEscape() throws IOException Handle an escape sequence. The current character must be the escape character. On return, the next character is available by calling `ExtendedBufferedReader#getLastChar()` on the input stream. Returns:int the unescaped character (as an int) or `IOUtils#EOF` if char following the escape is invalid. Exceptions IOException: if there is a problem reading the stream or the end of stream is detected: the escape character is not allowed at end of stream CSVException: Thrown on invalid input.

pack-priv int readEscape() throws IOException

Handle an escape sequence. The current character must be the escape character. On return, the next character is available by calling ExtendedBufferedReader#getLastChar() on the input stream.

Returns:int

the unescaped character (as an int) or IOUtils#EOF if char following the escape is invalid.

Exceptions

IOException:: if there is a problem reading the stream or the end of stream is detected: the escape character is not allowed at end of stream
CSVException:: Thrown on invalid input.

trimTrailingSpaces	back to summary
pack-priv void trimTrailingSpaces(final StringBuilder buffer)

pack-priv final Class Lexer

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail