Modifier and Type	Class and Description
pack-priv static class	DataFileReader.SeekableInputStream

Modifier and Type	Field and Description
private long	blockStart
private int[]	partialMatchTable
private DataFileReader.SeekableInputStream	sin

Access	Constructor and Description
public	DataFileReader(File file, DatumReader<D> reader) Construct a reader for a file.
public	DataFileReader(SeekableInput sin, DatumReader<D> reader) Construct a reader for a file.
private	DataFileReader(SeekableInput sin, DatumReader<D> reader, byte[] magic)
protected	DataFileReader(SeekableInput sin, DatumReader<D> reader, boolean closeOnError, byte[] magic) Construct a reader for a file.
protected	DataFileReader(SeekableInput sin, DatumReader<D> reader, DataFileStream.Header header) Construct using a `DataFileStream.Header`.

Modifier and Type	Method and Description
protected void	blockFinished() Overrides org.apache.avro.file.DataFileStream.blockFinished.
private int[]	Returns: the pre-computed partial match table computePartialMatchTable(final byte[] The pattern being searched pattern) Compute that Knuth-Morris-Pratt partial match table.
public static <D> FileReader<D>	openReader(File file, DatumReader<D> reader) Open a reader for a file.
public static <D> FileReader<D>	openReader(SeekableInput in, DatumReader<D> reader) Open a reader for a file.
public static <D> DataFileReader<D>	openReader(SeekableInput in, DatumReader<D> reader, DataFileStream.Header header, boolean True to read forward to the next sync point after opening, false to assume that the input is already at a valid sync point. sync) Construct a reader for a file at the current position of the input, without reading the header.
public boolean	pastSync(long position) Implements org.apache.avro.file.FileReader.pastSync. Return true if past the next synchronization point after a position.
public long	previousSync() Return the last synchronization point before our current position.
public void	seek(long position) Move to a specific, known synchronization point, one returned from `DataFileWriter#sync()` while writing.
public void	sync(final long position) Implements org.apache.avro.file.FileReader.sync. Move to the next synchronization point after a position.
public long	tell() Implements org.apache.avro.file.FileReader.tell. Return the current position in the input.

blockStart	back to summary
private long blockStart

partialMatchTable	back to summary
private int[] partialMatchTable

sin	back to summary
private DataFileReader.SeekableInputStream sin

DataFileReader	back to summary
public DataFileReader(File file, DatumReader<D> reader) throws IOException Construct a reader for a file. For example, if you want to read a file record,you need to close the resource. You can use try-with-resource as follows: try (FileReader<User> dataFileReader = DataFileReader.openReader(file,datumReader)) { //Consume the reader } catch (IOException e) { throw new RunTimeIOException(e,"Failed to read metadata for file: %s", file); }

DataFileReader	back to summary
public DataFileReader(SeekableInput sin, DatumReader<D> reader) throws IOException Construct a reader for a file. For example, if you want to read a file record,you need to close the resource. You can use try-with-resource as follows: try (FileReader<User> dataFileReader = DataFileReader.openReader(file,datumReader)) { //Consume the reader } catch (IOException e) { throw new RunTimeIOException(e,"Failed to read metadata for file: %s", file); }

DataFileReader	back to summary
private DataFileReader(SeekableInput sin, DatumReader<D> reader, byte[] magic) throws IOException

DataFileReader	back to summary
protected DataFileReader(SeekableInput sin, DatumReader<D> reader, boolean closeOnError, byte[] magic) throws IOException Construct a reader for a file. Please close resource files yourself.

DataFileReader	back to summary
protected DataFileReader(SeekableInput sin, DatumReader<D> reader, DataFileStream.Header header) throws IOException Construct using a `DataFileStream.Header`. Does not call `sync(long)` or `seek(long)`.

Method Detail

blockFinished	back to summary
protected void blockFinished() throws IOException Overrides org.apache.avro.file.DataFileStream.blockFinished. Annotations @Override

blockFinished

back to summary

protected void blockFinished() throws IOException

Overrides org.apache.avro.file.DataFileStream.blockFinished.

Annotations

@Override

computePartialMatchTable	back to summary
private int[] computePartialMatchTable(final byte[] pattern) Compute that Knuth-Morris-Pratt partial match table. Parameters pattern:byte[] The pattern being searched Returns:int[] the pre-computed partial match table See Also William Fiset Algorithms

computePartialMatchTable

back to summary

private int[] computePartialMatchTable(final byte[] pattern)

Compute that Knuth-Morris-Pratt partial match table.

Parameters

pattern:byte[]: The pattern being searched

Returns:int[]

the pre-computed partial match table

See Also

William Fiset Algorithms

openReader	back to summary
public static <D> FileReader<D> openReader(File file, DatumReader<D> reader) throws IOException Open a reader for a file.

openReader	back to summary
public static <D> FileReader<D> openReader(SeekableInput in, DatumReader<D> reader) throws IOException Open a reader for a file.

openReader	back to summary
public static <D> DataFileReader<D> openReader(SeekableInput in, DatumReader<D> reader, DataFileStream.Header header, boolean sync) throws IOException Construct a reader for a file at the current position of the input, without reading the header. Parameters sync:boolean True to read forward to the next sync point after opening, false to assume that the input is already at a valid sync point.

openReader

back to summary

public static <D> DataFileReader<D> openReader(SeekableInput in, DatumReader<D> reader, DataFileStream.Header header, boolean sync) throws IOException

Construct a reader for a file at the current position of the input, without reading the header.

Parameters

sync:boolean: True to read forward to the next sync point after opening, false to assume that the input is already at a valid sync point.

pastSync	back to summary
public boolean pastSync(long position) throws IOException Implements org.apache.avro.file.FileReader.pastSync. Return true if past the next synchronization point after a position. Annotations @Override

pastSync

back to summary

public boolean pastSync(long position) throws IOException

Implements org.apache.avro.file.FileReader.pastSync.

Return true if past the next synchronization point after a position.

Annotations

@Override

previousSync	back to summary
public long previousSync() Return the last synchronization point before our current position.

seek	back to summary
public void seek(long position) throws IOException Move to a specific, known synchronization point, one returned from `DataFileWriter#sync()` while writing. If synchronization points were not saved while writing a file, use `sync(long)` instead.

sync back to summary

sync	back to summary
public void sync(final long position) throws IOException Implements org.apache.avro.file.FileReader.sync. Move to the next synchronization point after a position. To process a range of file entires, call this with the starting position, then check `pastSync(long)` with the end point before each call to `next()`. Annotations @Override

public void sync(final long position) throws IOException

Implements org.apache.avro.file.FileReader.sync.

Move to the next synchronization point after a position. To process a range of file entires, call this with the starting position, then check pastSync(long) with the end point before each call to next().

Annotations

@Override

tell	back to summary
public long tell() throws IOException Implements org.apache.avro.file.FileReader.tell. Doc from org.apache.avro.file.FileReader.tell. Return the current position in the input. Annotations @Override

tell

back to summary

public long tell() throws IOException

Implements org.apache.avro.file.FileReader.tell.

Doc from org.apache.avro.file.FileReader.tell.

Return the current position in the input.

Annotations

@Override

org.apache.avro.file back to summary

pack-priv Class DataFileReader.SeekableInputStream

Class Inheritance

extends InputStream
implements SeekableInput

java.lang.Object
java.io.InputStream
org.apache.avro.file.DataFileReader.SeekableInputStream

All Implemented Interfaces

org.apache.avro.file.SeekableInput, java.io.Closeable, java.lang.AutoCloseable

Field Summary

Modifier and Type	Field and Description
private SeekableInput	in
private final byte[]	oneByte

Constructor Summary

Access	Constructor and Description
pack-priv	SeekableInputStream(SeekableInput in)

Method Summary

Modifier and Type	Method and Description
public int	available() Overrides java.io.InputStream.available. Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking, which may be 0, or 0 when end of stream is detected.
public void	close() Overrides java.io.InputStream.close. Implements java.io.Closeable.close. Closes this stream and releases any system resources associated with it.
public long	length() Implements org.apache.avro.file.SeekableInput.length. Return the length of the file.
public int	read(byte[] the buffer into which the data is read. b) Overrides java.io.InputStream.read. Reads some number of bytes from the input stream and stores them into the buffer array `b`.
public int	read(byte[] the buffer into which the data is read. b, int the start offset in array `b` at which the data is written. off, int the maximum number of bytes to read. len) Overrides java.io.InputStream.read. Implements org.apache.avro.file.SeekableInput.read. Equivalent to `java.io.InputStream#read(byte[], int, int)`.
public int	read() Implements abstract java.io.InputStream.read. Reads the next byte of data from the input stream.
public void	seek(long p) Implements org.apache.avro.file.SeekableInput.seek. Set the position for the next `read()`.
public long	skip(long the number of bytes to be skipped. skip) Overrides java.io.InputStream.skip. Skips over and discards `n` bytes of data from this input stream.
public long	tell() Implements org.apache.avro.file.SeekableInput.tell. Return the position of the next `read()`.

Inherited from java.io.InputStream:: mark markSupported nullInputStream readAllBytes readNBytes readNBytes reset skipNBytes transferTo

Field Detail

in	back to summary
private SeekableInput in

oneByte	back to summary
private final byte[] oneByte

Constructor Detail

SeekableInputStream	back to summary
pack-priv SeekableInputStream(SeekableInput in) throws IOException

Method Detail

available back to summary

available	back to summary
public int available() throws IOException Overrides java.io.InputStream.available. Doc from java.io.InputStream.available. Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking, which may be 0, or 0 when end of stream is detected. The read might be on the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes. Note that while some implementations of `InputStream` will return the total number of bytes in the stream, many will not. It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream. A subclass's implementation of this method may choose to throw an `IOException` if this input stream has been closed by invoking the `close()` method. Returns:int an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking or `0` when it reaches the end of the input stream. Annotations @Override Exceptions IOException: if an I/O error occurs.

public int available() throws IOException

Overrides java.io.InputStream.available.

Doc from java.io.InputStream.available.

Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking, which may be 0, or 0 when end of stream is detected. The read might be on the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes.

Note that while some implementations of InputStream will return the total number of bytes in the stream, many will not. It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream.

A subclass's implementation of this method may choose to throw an IOException if this input stream has been closed by invoking the close() method.

Returns:int

an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking or 0 when it reaches the end of the input stream.

Annotations

@Override

Exceptions

IOException:: if an I/O error occurs.

close back to summary

close	back to summary
public void close() throws IOException Overrides java.io.InputStream.close. Implements java.io.Closeable.close. Doc from java.io.Closeable.close. Closes this stream and releases any system resources associated with it. If the stream is already closed then invoking this method has no effect. As noted in `AutoCloseable#close()`, cases where the close may fail require careful attention. It is strongly advised to relinquish the underlying resources and to internally mark the `Closeable` as closed, prior to throwing the `IOException`. Annotations @Override Exceptions IOException: if an I/O error occurs

public void close() throws IOException

Overrides java.io.InputStream.close.

Implements java.io.Closeable.close.

Doc from java.io.Closeable.close.

Closes this stream and releases any system resources associated with it. If the stream is already closed then invoking this method has no effect.

As noted in AutoCloseable#close(), cases where the close may fail require careful attention. It is strongly advised to relinquish the underlying resources and to internally mark the Closeable as closed, prior to throwing the IOException.

Annotations

@Override

Exceptions

IOException:: if an I/O error occurs

length	back to summary
public long length() throws IOException Implements org.apache.avro.file.SeekableInput.length. Doc from org.apache.avro.file.SeekableInput.length. Return the length of the file. Annotations @Override

length

back to summary

public long length() throws IOException

Implements org.apache.avro.file.SeekableInput.length.

Doc from org.apache.avro.file.SeekableInput.length.

Return the length of the file.

Annotations

@Override

read back to summary

read	back to summary
public int read(byte[] b) throws IOException Overrides java.io.InputStream.read. Doc from java.io.InputStream.read. Reads some number of bytes from the input stream and stores them into the buffer array `b`. The number of bytes actually read is returned as an integer. This method blocks until input data is available, end of file is detected, or an exception is thrown. If the length of `b` is zero, then no bytes are read and `0` is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at the end of the file, the value `-1` is returned; otherwise, at least one byte is read and stored into `b`. The first byte read is stored into element `b[0]`, the next one into `b[1]`, and so on. The number of bytes read is, at most, equal to the length of `b`. Let k be the number of bytes actually read; these bytes will be stored in elements `b[0]` through `b[`k`-1]`, leaving elements `b[`k`]` through `b[b.length-1]` unaffected. Parameters b:byte[] the buffer into which the data is read. Returns:int the total number of bytes read into the buffer, or `-1` if there is no more data because the end of the stream has been reached. Annotations @Override Exceptions IOException: If the first byte cannot be read for any reason other than the end of the file, if the input stream has been closed, or if some other I/O error occurs.

public int read(byte[] b) throws IOException

Overrides java.io.InputStream.read.

Doc from java.io.InputStream.read.

Reads some number of bytes from the input stream and stores them into the buffer array b. The number of bytes actually read is returned as an integer. This method blocks until input data is available, end of file is detected, or an exception is thrown.

If the length of b is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at the end of the file, the value -1 is returned; otherwise, at least one byte is read and stored into b.

The first byte read is stored into element b[0], the next one into b[1], and so on. The number of bytes read is, at most, equal to the length of b. Let k be the number of bytes actually read; these bytes will be stored in elements b[0] through b[k-1], leaving elements b[k] through b[b.length-1] unaffected.

Parameters

b:byte[]: the buffer into which the data is read.

Returns:int

the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached.

Annotations

@Override

Exceptions

IOException:: If the first byte cannot be read for any reason other than the end of the file, if the input stream has been closed, or if some other I/O error occurs.

read back to summary

read	back to summary
public int read(byte[] b, int off, int len) throws IOException Overrides java.io.InputStream.read. Implements org.apache.avro.file.SeekableInput.read. Doc from org.apache.avro.file.SeekableInput.read. Equivalent to `java.io.InputStream#read(byte[], int, int)`. Parameters b:byte[] Doc from java.io.InputStream.read. the buffer into which the data is read. off:int Doc from java.io.InputStream.read. the start offset in array `b` at which the data is written. len:int Doc from java.io.InputStream.read. the maximum number of bytes to read. Returns:int Doc from java.io.InputStream.read. the total number of bytes read into the buffer, or `-1` if there is no more data because the end of the stream has been reached. Annotations @Override Exceptions IOException: Doc from java.io.InputStream.read. If the first byte cannot be read for any reason other than end of file, or if the input stream has been closed, or if some other I/O error occurs.

public int read(byte[] b, int off, int len) throws IOException

Overrides java.io.InputStream.read.

Implements org.apache.avro.file.SeekableInput.read.

Doc from org.apache.avro.file.SeekableInput.read.

Equivalent to java.io.InputStream#read(byte[], int, int).

Parameters

b:byte[]

Doc from java.io.InputStream.read.

the buffer into which the data is read.

off:int

Doc from java.io.InputStream.read.

the start offset in array b at which the data is written.

len:int

Doc from java.io.InputStream.read.

the maximum number of bytes to read.

Returns:int

Doc from java.io.InputStream.read.

the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached.

Annotations

@Override

Exceptions

IOException:

Doc from java.io.InputStream.read.

If the first byte cannot be read for any reason other than end of file, or if the input stream has been closed, or if some other I/O error occurs.

read back to summary

read	back to summary
public int read() throws IOException Implements abstract java.io.InputStream.read. Doc from java.io.InputStream.read. Reads the next byte of data from the input stream. The value byte is returned as an `int` in the range `0` to `255`. If no byte is available because the end of the stream has been reached, the value `-1` is returned. This method blocks until input data is available, the end of the stream is detected, or an exception is thrown. Returns:int the next byte of data, or `-1` if the end of the stream is reached. Annotations @Override Exceptions IOException: if an I/O error occurs.

public int read() throws IOException

Implements abstract java.io.InputStream.read.

Doc from java.io.InputStream.read.

Reads the next byte of data from the input stream. The value byte is returned as an int in the range 0 to 255. If no byte is available because the end of the stream has been reached, the value -1 is returned. This method blocks until input data is available, the end of the stream is detected, or an exception is thrown.

Returns:int

the next byte of data, or -1 if the end of the stream is reached.

Annotations

@Override

Exceptions

IOException:: if an I/O error occurs.

seek back to summary

seek	back to summary
public void seek(long p) throws IOException Implements org.apache.avro.file.SeekableInput.seek. Doc from org.apache.avro.file.SeekableInput.seek. Set the position for the next `read()`. Annotations @Override

public void seek(long p) throws IOException

Implements org.apache.avro.file.SeekableInput.seek.

Doc from org.apache.avro.file.SeekableInput.seek.

Set the position for the next read().

Annotations

@Override

skip back to summary

skip	back to summary
public long skip(long skip) throws IOException Overrides java.io.InputStream.skip. Doc from java.io.InputStream.skip. Skips over and discards `n` bytes of data from this input stream. The `skip` method may, for a variety of reasons, end up skipping over some smaller number of bytes, possibly `0`. This may result from any of a number of conditions; reaching end of file before `n` bytes have been skipped is only one possibility. The actual number of bytes skipped is returned. If `n` is negative, the `skip` method for class `InputStream` always returns 0, and no bytes are skipped. Subclasses may handle the negative value differently. Parameters skip:long the number of bytes to be skipped. Returns:long the actual number of bytes skipped which might be zero. Annotations @Override Exceptions IOException: if an I/O error occurs.

public long skip(long skip) throws IOException

Overrides java.io.InputStream.skip.

Doc from java.io.InputStream.skip.

Skips over and discards n bytes of data from this input stream. The skip method may, for a variety of reasons, end up skipping over some smaller number of bytes, possibly 0. This may result from any of a number of conditions; reaching end of file before n bytes have been skipped is only one possibility. The actual number of bytes skipped is returned. If n is negative, the skip method for class InputStream always returns 0, and no bytes are skipped. Subclasses may handle the negative value differently.

Parameters

skip:long: the number of bytes to be skipped.

Returns:long

the actual number of bytes skipped which might be zero.

Annotations

@Override

Exceptions

IOException:: if an I/O error occurs.

tell back to summary

tell	back to summary
public long tell() throws IOException Implements org.apache.avro.file.SeekableInput.tell. Doc from org.apache.avro.file.SeekableInput.tell. Return the position of the next `read()`. Annotations @Override

public long tell() throws IOException

Implements org.apache.avro.file.SeekableInput.tell.

Doc from org.apache.avro.file.SeekableInput.tell.

Return the position of the next read().

Annotations

@Override

public Class DataFileReader<D>

Nested and Inner Type Summary

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail

pack-priv Class DataFileReader.SeekableInputStream

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail