Top Description Inners Fields Constructors Methods
de.siegmar.fastcsv.reader

public final Class IndexedCsvReader<T>

extends Object
implements Closeable
Class Inheritance
All Implemented Interfaces
java.io.Closeable, java.lang.AutoCloseable
Annotations
@SuppressWarnings:checkstyle:ClassFanOutComplexity, checkstyle:ClassDataAbstractionCoupling
Type Parameters
<T>
the type of the CSV record.
Imports
java.io.Closeable, .IOException, .InputStreamReader, .RandomAccessFile, java.nio.charset.Charset, .StandardCharsets, java.nio.file.Files, .Path, .StandardOpenOption, java.util.ArrayList, .List, .Objects, .Optional, .StringJoiner, java.util.concurrent.atomic.AtomicLong, java.util.concurrent.locks.Lock, .ReentrantLock, de.siegmar.fastcsv.util.Nullable, .Preconditions, .Util
CSV reader implementation for indexed based access.

If no prebuilt index passed in (via IndexedCsvReaderBuilder#index(CsvIndex)) the constructor will initiate indexing the file. This process is optimized on performance and low memory usage – no CSV data is stored in memory. The current status can be monitored via IndexedCsvReaderBuilder#statusListener(StatusListener).

This class is thread-safe.

Example use:

try (IndexedCsvReader&lt;CsvRecord&gt; csv = IndexedCsvReader.builder().ofCsvRecord(file)){ CsvIndex index = csv.getIndex(); int lastPage = index.getPageCount() - 1; List&lt;CsvRecord&gt; csvRecords = csv.readPage(lastPage); } 
try (IndexedCsvReader&lt;CsvRecord&gt; csv = IndexedCsvReader.builder().ofCsvRecord(file)){
    CsvIndex index = csv.getIndex();
    int lastPage = index.getPageCount() - 1;
    List&lt;CsvRecord&gt; csvRecords = csv.readPage(lastPage);
}

Nested and Inner Type Summary

Modifier and TypeClass and Description
public static class
IndexedCsvReader.IndexedCsvReaderBuilder

This builder is used to create configured instances of IndexedCsvReader.

private class
IndexedCsvReader.ScannerListener

Field Summary

Modifier and TypeField and Description
private final boolean
allowExtraCharsAfterClosingQuote

private final Charset
charset

private final char
commentCharacter

private final CommentStrategy
commentStrategy

private final CsvIndex
csvIndex

private final CsvParser
csvParser

private final CsvCallbackHandler<T>
csvRecordHandler

private final char
fieldSeparator

private final Path
file

private final Lock
fileLock

private final int
pageSize

private final char
quoteCharacter

private final RandomAccessFile
raf

Constructor Summary

AccessConstructor and Description
pack-priv
IndexedCsvReader(final Path file, final Charset defaultCharset, final char fieldSeparator, final char quoteCharacter, final CommentStrategy commentStrategy, final char commentCharacter, final boolean allowExtraCharsAfterClosingQuote, final int maxBufferSize, final int pageSize, final CsvCallbackHandler<T> csvRecordHandler, final CsvIndex csvIndex, final StatusListener statusListener)

Method Summary

Modifier and TypeMethod and Description
private static void
assertFields(final char fieldSeparator, final char quoteCharacter, final char commentCharacter, final CommentStrategy commentStrategy)

public static IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

a new IndexedCsvReaderBuilder instance.
builder()

Constructs a IndexedCsvReaderBuilder to configure and build instances of this class.

private String
buildExceptionMessage()

private CsvIndex
buildIndex(final int bomHeaderLength, final StatusListener statusListener)

public void
close()

Implements java.io.Closeable.close.

Closes this stream and releases any system resources associated with it.

private static Optional<BomHeader>
detectBom(final Path file, final StatusListener statusListener)

public CsvIndex

Returns:

the index that is used for accessing the CSV file.
getIndex()

Get the index used for accessing the CSV file.

public List<T>

Returns:

a page of records, never null.
readPage(final int
the page to read (0-based).
page)

Reads a page of records.

private List<T>
readPage(final CsvIndex.CsvPage page)

public String
toString()

Overrides java.lang.Object.toString.

Returns a string representation of the object.

private static CsvIndex
validatePrebuiltIndex(final Path file, final int bomHeaderLength, final byte fieldSeparator, final byte quoteCharacter, final CommentStrategy commentStrategy, final byte commentCharacter, final CsvIndex csvIndex)

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAllwaitwaitwait

Field Detail

allowExtraCharsAfterClosingQuoteback to summary
private final boolean allowExtraCharsAfterClosingQuote
charsetback to summary
private final Charset charset
commentCharacterback to summary
private final char commentCharacter
commentStrategyback to summary
private final CommentStrategy commentStrategy
csvIndexback to summary
private final CsvIndex csvIndex
csvParserback to summary
private final CsvParser csvParser
csvRecordHandlerback to summary
private final CsvCallbackHandler<T> csvRecordHandler
fieldSeparatorback to summary
private final char fieldSeparator
fileback to summary
private final Path file
fileLockback to summary
private final Lock fileLock
pageSizeback to summary
private final int pageSize
quoteCharacterback to summary
private final char quoteCharacter
rafback to summary
private final RandomAccessFile raf

Constructor Detail

IndexedCsvReaderback to summary
pack-priv IndexedCsvReader(final Path file, final Charset defaultCharset, final char fieldSeparator, final char quoteCharacter, final CommentStrategy commentStrategy, final char commentCharacter, final boolean allowExtraCharsAfterClosingQuote, final int maxBufferSize, final int pageSize, final CsvCallbackHandler<T> csvRecordHandler, final CsvIndex csvIndex, final StatusListener statusListener) throws IOException
Annotations
@SuppressWarnings:checkstyle:ParameterNumber

Method Detail

assertFieldsback to summary
private static void assertFields(final char fieldSeparator, final char quoteCharacter, final char commentCharacter, final CommentStrategy commentStrategy)
builderback to summary
public static IndexedCsvReader.IndexedCsvReaderBuilder builder()

Constructs a IndexedCsvReaderBuilder to configure and build instances of this class.

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

a new IndexedCsvReaderBuilder instance.

buildExceptionMessageback to summary
private String buildExceptionMessage()
buildIndexback to summary
private CsvIndex buildIndex(final int bomHeaderLength, final StatusListener statusListener) throws IOException
Annotations
@SuppressWarnings:checkstyle:IllegalCatch, PMD.AvoidCatchingThrowable
closeback to summary
public void close() throws IOException

Implements java.io.Closeable.close.

Doc from java.io.Closeable.close.

Closes this stream and releases any system resources associated with it. If the stream is already closed then invoking this method has no effect.

As noted in AutoCloseable#close(), cases where the close may fail require careful attention. It is strongly advised to relinquish the underlying resources and to internally mark the Closeable as closed, prior to throwing the IOException.

Annotations
@Override
Exceptions
IOException:
if an I/O error occurs
detectBomback to summary
private static Optional<BomHeader> detectBom(final Path file, final StatusListener statusListener) throws IOException
getIndexback to summary
public CsvIndex getIndex()

Get the index used for accessing the CSV file. That index is either a freshly built index or the index that has been passed via IndexedCsvReaderBuilder#index(CsvIndex).

Returns:CsvIndex

the index that is used for accessing the CSV file.

readPageback to summary
public List<T> readPage(final int page) throws IOException

Reads a page of records.

Parameters
page:int

the page to read (0-based).

Returns:List<T>

a page of records, never null.

Exceptions
IOException:
if an I/O error occurs.
IllegalArgumentException:
if page is < 0
IndexOutOfBoundsException:
if the file does not contain the specified page
readPageback to summary
private List<T> readPage(final CsvIndex.CsvPage page) throws IOException
Annotations
@SuppressWarnings:checkstyle:IllegalCatch, PMD.AvoidCatchingThrowable
toStringback to summary
public String toString()

Overrides java.lang.Object.toString.

Doc from java.lang.Object.toString.

Returns a string representation of the object. Satisfying this method's contract implies a non-null result must be returned.

Annotations
@Override
validatePrebuiltIndexback to summary
private static CsvIndex validatePrebuiltIndex(final Path file, final int bomHeaderLength, final byte fieldSeparator, final byte quoteCharacter, final CommentStrategy commentStrategy, final byte commentCharacter, final CsvIndex csvIndex) throws IOException
de.siegmar.fastcsv.reader back to summary

public final Class IndexedCsvReader.IndexedCsvReaderBuilder

extends Object
Class Inheritance
  • java.lang.Object
  • de.siegmar.fastcsv.reader.IndexedCsvReader.IndexedCsvReaderBuilder
Annotations
@SuppressWarnings:checkstyle:HiddenField, PMD.AvoidFieldNameMatchingMethodName
This builder is used to create configured instances of IndexedCsvReader. The default configuration of this class adheres with RFC 4180:

The line delimiter (line-feed, carriage-return or the combination of both) is detected automatically and thus not configurable.

Field Summary

Modifier and TypeField and Description
private boolean
allowExtraCharsAfterClosingQuote

private char
commentCharacter

private CommentStrategy
commentStrategy

private CsvIndex
csvIndex

private static final int
DEFAULT_MAX_BUFFER_SIZE

private static final int
DEFAULT_PAGE_SIZE

private char
fieldSeparator

private static final int
MAX_BASE_ASCII

private int
maxBufferSize

private static final int
MIN_PAGE_SIZE

private int
pageSize

private char
quoteCharacter

private StatusListener
statusListener

Constructor Summary

AccessConstructor and Description
private
IndexedCsvReaderBuilder()

Method Summary

Modifier and TypeMethod and Description
public IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

This updated object, allowing additional method calls to be chained together.
allowExtraCharsAfterClosingQuote(final boolean
allow extra characters after closing quotes (default: false).
allowExtraCharsAfterClosingQuote)

Specifies whether the presence of characters between a closing quote and a field separator or the end of a line should be treated as an error or not.

public <
the type of the CSV record.
T> IndexedCsvReader<T>

Returns:

a new IndexedCsvReader - never null. Remember to close it!
build(final CsvCallbackHandler<T>
the callback handler to use.
callbackHandler, final Path
the file to read data from.
file)

Constructs a new IndexedCsvReader for the specified callback handler and path using UTF-8 as the character set.

public <
the type of the CSV record.
T> IndexedCsvReader<T>

Returns:

a new IndexedCsvReader - never null. Remember to close it!
build(final CsvCallbackHandler<T>
the callback handler to use.
callbackHandler, final Path
the file to read data from.
file, final Charset
the character set to use.
charset)

Constructs a new IndexedCsvReader for the specified arguments.

private static void
checkControlCharacter(final char controlChar)

public IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

This updated object, allowing additional method calls to be chained together.
commentCharacter(final char
the character used to comment lines (default: # - hash)
commentCharacter)

Sets the commentCharacter used to comment lines.

public IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

This updated object, allowing additional method calls to be chained together.
commentStrategy(final CommentStrategy
the strategy for handling comments.
commentStrategy)

Sets the strategy that defines how (and if) commented lines should be handled (default: CommentStrategy#NONE as comments are not defined in RFC 4180).

public IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

This updated object, allowing additional method calls to be chained together.
fieldSeparator(final char
the field separator character (default: , - comma).
fieldSeparator)

Sets the fieldSeparator used when reading CSV data.

public IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

This updated object, allowing additional method calls to be chained together.
index(final CsvIndex
a prebuilt index
csvIndex)

Sets a prebuilt index that should be used for accessing the file.

public IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

This updated object, allowing additional method calls to be chained together.
maxBufferSize(final int
the maximum buffer size in characters (default: 16777216)
maxBufferSize)

Defines the maximum buffer size used when parsing data.

public IndexedCsvReader<CsvRecord>

Returns:

a new IndexedCsvReader - never null. Remember to close it!
ofCsvRecord(final Path
the file to read data from.
file)

Constructs a new IndexedCsvReader of CsvRecord for the specified path using UTF-8 as the character set.

public IndexedCsvReader<CsvRecord>

Returns:

a new IndexedCsvReader - never null. Remember to close it!
ofCsvRecord(final Path
the file to read data from.
file, final Charset
the character set to use.
charset)

Constructs a new IndexedCsvReader of CsvRecord for the specified arguments.

public IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

This updated object, allowing additional method calls to be chained together.
pageSize(final int
the maximum size of pages.
pageSize)

Sets the pageSize for pages returned by readPage(int) (default: DEFAULT_PAGE_SIZE).

public IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

This updated object, allowing additional method calls to be chained together.
quoteCharacter(final char
the character used to enclose fields (default: " - double quotes).
quoteCharacter)

Sets the quoteCharacter used when reading CSV data.

public IndexedCsvReader.IndexedCsvReaderBuilder

Returns:

This updated object, allowing additional method calls to be chained together.
statusListener(final StatusListener
the status listener.
statusListener)

Sets the statusListener to listen for indexer status updates.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

allowExtraCharsAfterClosingQuoteback to summary
private boolean allowExtraCharsAfterClosingQuote
commentCharacterback to summary
private char commentCharacter
commentStrategyback to summary
private CommentStrategy commentStrategy
csvIndexback to summary
private CsvIndex csvIndex
Annotations
@Nullable
DEFAULT_MAX_BUFFER_SIZEback to summary
private static final int DEFAULT_MAX_BUFFER_SIZE
DEFAULT_PAGE_SIZEback to summary
private static final int DEFAULT_PAGE_SIZE
fieldSeparatorback to summary
private char fieldSeparator
MAX_BASE_ASCIIback to summary
private static final int MAX_BASE_ASCII
maxBufferSizeback to summary
private int maxBufferSize
MIN_PAGE_SIZEback to summary
private static final int MIN_PAGE_SIZE
pageSizeback to summary
private int pageSize
quoteCharacterback to summary
private char quoteCharacter
statusListenerback to summary
private StatusListener statusListener
Annotations
@Nullable

Constructor Detail

IndexedCsvReaderBuilderback to summary
private IndexedCsvReaderBuilder()

Method Detail

allowExtraCharsAfterClosingQuoteback to summary
public IndexedCsvReader.IndexedCsvReaderBuilder allowExtraCharsAfterClosingQuote(final boolean allowExtraCharsAfterClosingQuote)

Specifies whether the presence of characters between a closing quote and a field separator or the end of a line should be treated as an error or not.

Example: "a"b,"c"

If this is set to true, the value ab will be returned for the first field.

If this is set to false, a CsvParseException will be thrown.

Parameters
allowExtraCharsAfterClosingQuote:boolean

allow extra characters after closing quotes (default: false).

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

This updated object, allowing additional method calls to be chained together.

buildback to summary
public <T> IndexedCsvReader<T> build(final CsvCallbackHandler<T> callbackHandler, final Path file) throws IOException

Constructs a new IndexedCsvReader for the specified callback handler and path using UTF-8 as the character set.

Convenience method for build(CsvCallbackHandler, Path, Charset) with StandardCharsets#UTF_8 as charset.

Parameters
<T>
the type of the CSV record.
callbackHandler:CsvCallbackHandler<T>

the callback handler to use.

file:Path

the file to read data from.

Returns:IndexedCsvReader<T>

a new IndexedCsvReader - never null. Remember to close it!

Exceptions
IOException:
if an I/O error occurs.
NullPointerException:
if callbackHandler, file or charset is null
buildback to summary
public <T> IndexedCsvReader<T> build(final CsvCallbackHandler<T> callbackHandler, final Path file, final Charset charset) throws IOException

Constructs a new IndexedCsvReader for the specified arguments.

Parameters
<T>
the type of the CSV record.
callbackHandler:CsvCallbackHandler<T>

the callback handler to use.

file:Path

the file to read data from.

charset:Charset

the character set to use.

Returns:IndexedCsvReader<T>

a new IndexedCsvReader - never null. Remember to close it!

Exceptions
IOException:
if an I/O error occurs.
NullPointerException:
if callbackHandler, file or charset is null
IllegalArgumentException:
if argument validation fails.
checkControlCharacterback to summary
private static void checkControlCharacter(final char controlChar)
commentCharacterback to summary
public IndexedCsvReader.IndexedCsvReaderBuilder commentCharacter(final char commentCharacter)

Sets the commentCharacter used to comment lines.

Parameters
commentCharacter:char

the character used to comment lines (default: # - hash)

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

This updated object, allowing additional method calls to be chained together.

See Also
commentStrategy(CommentStrategy)
commentStrategyback to summary
public IndexedCsvReader.IndexedCsvReaderBuilder commentStrategy(final CommentStrategy commentStrategy)

Sets the strategy that defines how (and if) commented lines should be handled (default: CommentStrategy#NONE as comments are not defined in RFC 4180).

Parameters
commentStrategy:CommentStrategy

the strategy for handling comments.

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

This updated object, allowing additional method calls to be chained together.

Exceptions
IllegalArgumentException:
if CommentStrategy#SKIP is passed, as this is not supported
See Also
commentCharacter(char)
fieldSeparatorback to summary
public IndexedCsvReader.IndexedCsvReaderBuilder fieldSeparator(final char fieldSeparator)

Sets the fieldSeparator used when reading CSV data.

Parameters
fieldSeparator:char

the field separator character (default: , - comma).

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

This updated object, allowing additional method calls to be chained together.

indexback to summary
public IndexedCsvReader.IndexedCsvReaderBuilder index(final CsvIndex csvIndex)

Sets a prebuilt index that should be used for accessing the file.

Parameters
csvIndex:CsvIndex

a prebuilt index

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

This updated object, allowing additional method calls to be chained together.

maxBufferSizeback to summary
public IndexedCsvReader.IndexedCsvReaderBuilder maxBufferSize(final int maxBufferSize)

Defines the maximum buffer size used when parsing data.

The size of the internal buffer is automatically adjusted to the needs of the parser. To protect against out-of-memory errors, its maximum size is limited.

The buffer is used for two purposes:

  • Reading data from the underlying stream of data in chunks
  • Storing the data of a single field before it is passed to the callback handler

Set a larger value only if you expect to read fields larger than the default limit. In that case you probably also need to adjust the maximum field size of the callback handler.

Set a smaller value if your runtime environment has not enough memory available for the default value. Setting values smaller than 16,384 characters will most likely lead to performance degradation.

Parameters
maxBufferSize:int

the maximum buffer size in characters (default: 16777216)

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

This updated object, allowing additional method calls to be chained together.

Exceptions
IllegalArgumentException:
if maxBufferSize is not positive
ofCsvRecordback to summary
public IndexedCsvReader<CsvRecord> ofCsvRecord(final Path file) throws IOException

Constructs a new IndexedCsvReader of CsvRecord for the specified path using UTF-8 as the character set.

Convenience method for build(CsvCallbackHandler, Path, Charset) with CsvRecordHandler as the callback handler and StandardCharsets#UTF_8 as the charset.

Parameters
file:Path

the file to read data from.

Returns:IndexedCsvReader<CsvRecord>

a new IndexedCsvReader - never null. Remember to close it!

Exceptions
IOException:
if an I/O error occurs.
NullPointerException:
if file or charset is null
ofCsvRecordback to summary
public IndexedCsvReader<CsvRecord> ofCsvRecord(final Path file, final Charset charset) throws IOException

Constructs a new IndexedCsvReader of CsvRecord for the specified arguments.

Convenience method for build(CsvCallbackHandler, Path, Charset) with CsvRecordHandler as the callback handler.

Parameters
file:Path

the file to read data from.

charset:Charset

the character set to use.

Returns:IndexedCsvReader<CsvRecord>

a new IndexedCsvReader - never null. Remember to close it!

Exceptions
IOException:
if an I/O error occurs.
NullPointerException:
if file or charset is null
pageSizeback to summary
public IndexedCsvReader.IndexedCsvReaderBuilder pageSize(final int pageSize)

Sets the pageSize for pages returned by readPage(int) (default: DEFAULT_PAGE_SIZE).

Parameters
pageSize:int

the maximum size of pages.

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

This updated object, allowing additional method calls to be chained together.

quoteCharacterback to summary
public IndexedCsvReader.IndexedCsvReaderBuilder quoteCharacter(final char quoteCharacter)

Sets the quoteCharacter used when reading CSV data.

Parameters
quoteCharacter:char

the character used to enclose fields (default: " - double quotes).

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

This updated object, allowing additional method calls to be chained together.

statusListenerback to summary
public IndexedCsvReader.IndexedCsvReaderBuilder statusListener(final StatusListener statusListener)

Sets the statusListener to listen for indexer status updates.

Parameters
statusListener:StatusListener

the status listener.

Returns:IndexedCsvReader.IndexedCsvReaderBuilder

This updated object, allowing additional method calls to be chained together.

de.siegmar.fastcsv.reader back to summary

private final Class IndexedCsvReader.ScannerListener

extends Object
implements CsvListener
Class Inheritance
All Implemented Interfaces
de.siegmar.fastcsv.reader.CsvScanner.CsvListener

Field Summary

Modifier and TypeField and Description
private final List<CsvIndex.CsvPage>
pageOffsets

private final AtomicLong
recordCounter

private final AtomicLong
startingLineNumber

private final StatusListener
statusListener

Constructor Summary

AccessConstructor and Description
private
ScannerListener(final StatusListener statusListener)

Method Summary

Modifier and TypeMethod and Description
public void
additionalLine()

Implements de.siegmar.fastcsv.reader.CsvScanner.CsvListener.additionalLine.

public void
onReadBytes(final int readCnt)

Implements de.siegmar.fastcsv.reader.CsvScanner.CsvListener.onReadBytes.

public void
onReadRecord()

Implements de.siegmar.fastcsv.reader.CsvScanner.CsvListener.onReadRecord.

public void
startOffset(final long offset)

Implements de.siegmar.fastcsv.reader.CsvScanner.CsvListener.startOffset.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

pageOffsetsback to summary
private final List<CsvIndex.CsvPage> pageOffsets
recordCounterback to summary
private final AtomicLong recordCounter
startingLineNumberback to summary
private final AtomicLong startingLineNumber
statusListenerback to summary
private final StatusListener statusListener

Constructor Detail

ScannerListenerback to summary
private ScannerListener(final StatusListener statusListener)

Method Detail

additionalLineback to summary
public void additionalLine()

Implements de.siegmar.fastcsv.reader.CsvScanner.CsvListener.additionalLine.

Annotations
@Override
onReadBytesback to summary
public void onReadBytes(final int readCnt)

Implements de.siegmar.fastcsv.reader.CsvScanner.CsvListener.onReadBytes.

Annotations
@Override
onReadRecordback to summary
public void onReadRecord()

Implements de.siegmar.fastcsv.reader.CsvScanner.CsvListener.onReadRecord.

Annotations
@Override
startOffsetback to summary
public void startOffset(final long offset)

Implements de.siegmar.fastcsv.reader.CsvScanner.CsvListener.startOffset.

Annotations
@Override