Top Description Fields Constructors Methods
com.univocity.parsers.common.processor.core

public abstract Class AbstractBatchedColumnProcessor<T extends Context>

extends Object
implements Processor<T extends Context>, BatchedColumnReader<String>
Class Inheritance
All Implemented Interfaces
com.univocity.parsers.common.processor.core.BatchedColumnReader, com.univocity.parsers.common.processor.core.ColumnReader, com.univocity.parsers.common.processor.core.Processor
Known Direct Subclasses
com.univocity.parsers.common.processor.BatchedColumnProcessor
Imports
com.univocity.parsers.common.*, java.util.*
A Processor implementation that stores values of columns in batches. Use this implementation in favor of AbstractColumnProcessor when processing large inputs to avoid running out of memory. Values parsed in each row will be split into columns of Strings. Each column has its own list of values.

During the execution of the process, the batchProcessed(int) method will be invoked after a given number of rows has been processed.

The user can access the lists with values parsed for all columns using the methods getColumnValuesAsList(), getColumnValuesAsMapOfIndexes() and getColumnValuesAsMapOfNames().

After batchProcessed(int) is invoked, all values will be discarded and the next batch of column values will be accumulated. This process will repeat until there's no more rows in the input.

Author
Univocity Software Pty Ltd - [email protected]
See Also
AbstractParser, BatchedColumnReader, Processor

Field Summary

Modifier and TypeField and Description
private int
batchCount

private int
batchesProcessed

private final int
rowsPerBatch

private final ColumnSplitter<String>
splitter

Constructor Summary

AccessConstructor and Description
public
AbstractBatchedColumnProcessor(int
the number of rows to process in each batch.
rowsPerBatch)

Constructs a batched column processor configured to invoke the batchesProcessed method after a given number of rows has been processed.

Method Summary

Modifier and TypeMethod and Description
public abstract void
batchProcessed(int
the number of rows processed in the current batch. This corresponds to the number of elements of each list of each column.
rowsInThisBatch)

Redeclares com.univocity.parsers.common.processor.core.BatchedColumnReader.batchProcessed.

Callback to the user, where the lists with values parsed for all columns can be accessed using the methods getColumnValuesAsList(), getColumnValuesAsMapOfIndexes() and getColumnValuesAsMapOfNames().
public int
getBatchesProcessed()

Implements com.univocity.parsers.common.processor.core.BatchedColumnReader.getBatchesProcessed.

Returns the number of batches already processed
public List<String>
getColumn(String
the name of the column in the input.
columnName)

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumn.

Returns the values of a given column.
public List<String>
getColumn(int
the position of the column in the input (0-based).
columnIndex)

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumn.

Returns the values of a given column.
public final List<List<String>>
getColumnValuesAsList()

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumnValuesAsList.

Returns the values processed for each column
public final Map<Integer, List<String>>
getColumnValuesAsMapOfIndexes()

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumnValuesAsMapOfIndexes.

Returns a map of column indexes and their respective list of values parsed from the input.
public final Map<String, List<String>>
getColumnValuesAsMapOfNames()

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumnValuesAsMapOfNames.

Returns a map of column names and their respective list of values parsed from the input.
public final String[]
getHeaders()

Implements com.univocity.parsers.common.processor.core.ColumnReader.getHeaders.

Returns the column headers.
public int
getRowsPerBatch()

Implements com.univocity.parsers.common.processor.core.BatchedColumnReader.getRowsPerBatch.

Returns the number of rows processed in each batch
public void
processEnded(T
A contextual object with information and controls over the state of the parsing process
context)

Implements com.univocity.parsers.common.processor.core.Processor.processEnded.

This method will by invoked by the parser once, after the parsing process stopped and all resources were closed.
public void
processStarted(T
A contextual object with information and controls over the current state of the parsing process
context)

Implements com.univocity.parsers.common.processor.core.Processor.processStarted.

This method will by invoked by the parser once, when it is ready to start processing the input.
public final void
putColumnValuesInMapOfIndexes(Map<Integer, List<String>>
the map to hold the values of each column
map)

Implements com.univocity.parsers.common.processor.core.ColumnReader.putColumnValuesInMapOfIndexes.

Fills a given map associating each column index to its list of values
public final void
putColumnValuesInMapOfNames(Map<String, List<String>>
the map to hold the values of each column
map)

Implements com.univocity.parsers.common.processor.core.ColumnReader.putColumnValuesInMapOfNames.

Fills a given map associating each column name to its list o values
public void
rowProcessed(String[]
the data extracted by the parser for an individual record. Note that:
row, T
A contextual object with information and controls over the current state of the parsing process
context)

Implements com.univocity.parsers.common.processor.core.Processor.rowProcessed.

Invoked by the parser after all values of a valid record have been processed.
Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

batchCountback to summary
private int batchCount
batchesProcessedback to summary
private int batchesProcessed
rowsPerBatchback to summary
private final int rowsPerBatch
splitterback to summary
private final ColumnSplitter<String> splitter

Constructor Detail

AbstractBatchedColumnProcessorback to summary
public AbstractBatchedColumnProcessor(int rowsPerBatch)

Constructs a batched column processor configured to invoke the batchesProcessed method after a given number of rows has been processed.

Parameters
rowsPerBatch:int

the number of rows to process in each batch.

Method Detail

batchProcessedback to summary
public abstract void batchProcessed(int rowsInThisBatch)

Redeclares com.univocity.parsers.common.processor.core.BatchedColumnReader.batchProcessed.

Doc from com.univocity.parsers.common.processor.core.BatchedColumnReader.batchProcessed.

Callback to the user, where the lists with values parsed for all columns can be accessed using the methods getColumnValuesAsList(), getColumnValuesAsMapOfIndexes() and getColumnValuesAsMapOfNames().

Parameters
rowsInThisBatch:int

the number of rows processed in the current batch. This corresponds to the number of elements of each list of each column.

Annotations
@Override
getBatchesProcessedback to summary
public int getBatchesProcessed()

Implements com.univocity.parsers.common.processor.core.BatchedColumnReader.getBatchesProcessed.

Doc from com.univocity.parsers.common.processor.core.BatchedColumnReader.getBatchesProcessed.

Returns the number of batches already processed

Returns:int

the number of batches already processed

Annotations
@Override
getColumnback to summary
public List<String> getColumn(String columnName)

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumn.

Doc from com.univocity.parsers.common.processor.core.ColumnReader.getColumn.

Returns the values of a given column.

Parameters
columnName:String

the name of the column in the input.

Returns:List<String>

a list with all data stored in the given column

Annotations
@Override
getColumnback to summary
public List<String> getColumn(int columnIndex)

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumn.

Doc from com.univocity.parsers.common.processor.core.ColumnReader.getColumn.

Returns the values of a given column.

Parameters
columnIndex:int

the position of the column in the input (0-based).

Returns:List<String>

a list with all data stored in the given column

Annotations
@Override
getColumnValuesAsListback to summary
public final List<List<String>> getColumnValuesAsList()

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumnValuesAsList.

Doc from com.univocity.parsers.common.processor.core.ColumnReader.getColumnValuesAsList.

Returns the values processed for each column

Returns:List<List<String>>

a list of lists. The stored lists correspond to the position of the column processed from the input; Each list contains the corresponding values parsed for a column, across multiple rows.

Annotations
@Override
getColumnValuesAsMapOfIndexesback to summary
public final Map<Integer, List<String>> getColumnValuesAsMapOfIndexes()

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumnValuesAsMapOfIndexes.

Doc from com.univocity.parsers.common.processor.core.ColumnReader.getColumnValuesAsMapOfIndexes.

Returns a map of column indexes and their respective list of values parsed from the input.

Returns:Map<Integer, List<String>>

a map of column indexes and their respective list of values.

Annotations
@Override
getColumnValuesAsMapOfNamesback to summary
public final Map<String, List<String>> getColumnValuesAsMapOfNames()

Implements com.univocity.parsers.common.processor.core.ColumnReader.getColumnValuesAsMapOfNames.

Doc from com.univocity.parsers.common.processor.core.ColumnReader.getColumnValuesAsMapOfNames.

Returns a map of column names and their respective list of values parsed from the input.

Returns:Map<String, List<String>>

a map of column names and their respective list of values.

Annotations
@Override
getHeadersback to summary
public final String[] getHeaders()

Implements com.univocity.parsers.common.processor.core.ColumnReader.getHeaders.

Doc from com.univocity.parsers.common.processor.core.ColumnReader.getHeaders.

Returns the column headers. This can be either the headers defined in CommonSettings#getHeaders() or the headers parsed in the input when CommonSettings#getHeaders() equals to true

Returns:String[]

the headers of all column parsed.

Annotations
@Override
getRowsPerBatchback to summary
public int getRowsPerBatch()

Implements com.univocity.parsers.common.processor.core.BatchedColumnReader.getRowsPerBatch.

Doc from com.univocity.parsers.common.processor.core.BatchedColumnReader.getRowsPerBatch.

Returns the number of rows processed in each batch

Returns:int

the number of rows per batch

Annotations
@Override
processEndedback to summary
public void processEnded(T context)

Implements com.univocity.parsers.common.processor.core.Processor.processEnded.

Doc from com.univocity.parsers.common.processor.core.Processor.processEnded.

This method will by invoked by the parser once, after the parsing process stopped and all resources were closed.

It will always be called by the parser: in case of errors, if the end of the input us reached, or if the user stopped the process manually using ParsingContext#stop().

Parameters
context:T

A contextual object with information and controls over the state of the parsing process

Annotations
@Override
processStartedback to summary
public void processStarted(T context)

Implements com.univocity.parsers.common.processor.core.Processor.processStarted.

Doc from com.univocity.parsers.common.processor.core.Processor.processStarted.

This method will by invoked by the parser once, when it is ready to start processing the input.

Parameters
context:T

A contextual object with information and controls over the current state of the parsing process

Annotations
@Override
putColumnValuesInMapOfIndexesback to summary
public final void putColumnValuesInMapOfIndexes(Map<Integer, List<String>> map)

Implements com.univocity.parsers.common.processor.core.ColumnReader.putColumnValuesInMapOfIndexes.

Doc from com.univocity.parsers.common.processor.core.ColumnReader.putColumnValuesInMapOfIndexes.

Fills a given map associating each column index to its list of values

Parameters
map:Map<Integer, List<String>>

the map to hold the values of each column

Annotations
@Override
putColumnValuesInMapOfNamesback to summary
public final void putColumnValuesInMapOfNames(Map<String, List<String>> map)

Implements com.univocity.parsers.common.processor.core.ColumnReader.putColumnValuesInMapOfNames.

Doc from com.univocity.parsers.common.processor.core.ColumnReader.putColumnValuesInMapOfNames.

Fills a given map associating each column name to its list o values

Parameters
map:Map<String, List<String>>

the map to hold the values of each column

Annotations
@Override
rowProcessedback to summary
public void rowProcessed(String[] row, T context)

Implements com.univocity.parsers.common.processor.core.Processor.rowProcessed.

Doc from com.univocity.parsers.common.processor.core.Processor.rowProcessed.

Invoked by the parser after all values of a valid record have been processed.

Parameters
row:String[]

the data extracted by the parser for an individual record. Note that:

context:T

A contextual object with information and controls over the current state of the parsing process

Annotations
@Override