Modifier and Type	Class and Description
pack-priv static class	DataFileStream.DataBlock
public static class	DataFileStream.Header A handle that can be used to reopen a DataFile without re-reading the header of the stream.

Field Summary

Modifier and Type	Field and Description
private boolean	availableBlock
private DataFileStream.DataBlock	block
pack-priv ByteBuffer	blockBuffer
pack-priv long	blockCount
pack-priv long	blockRemaining
private long	blockSize
private Codec	codec
pack-priv BinaryDecoder	datumIn Secondary decoder, for datums.
private DataFileStream.Header	header
private DatumReader<D>	reader
pack-priv byte[]	syncBuffer
pack-priv BinaryDecoder	vin Decoder on raw input stream.

Constructor Summary

Access	Constructor and Description
public	DataFileStream(InputStream in, DatumReader<D> reader) Construct a reader for an input stream.
protected	DataFileStream(DatumReader<D> reader) create an uninitialized DataFileStream

Method Summary

Modifier and Type	Method and Description
protected void	blockFinished()
public void	close() Implements java.io.Closeable.close. Close this reader.
public long	getBlockCount() Expert: Return the count of items in the current block.
public long	getBlockSize() Expert: Return the size in bytes (uncompressed) of the current block.
public DataFileStream.Header	getHeader() A handle that can be used to reopen this stream without rereading the head.
public byte[]	getMeta(String key) Return the value of a metadata property.
public List<String>	getMetaKeys() Return the list of keys in the metadata
public long	getMetaLong(String key) Return the value of a metadata property.
public String	getMetaString(String key) Return the value of a metadata property.
public Schema	getSchema() Return the schema used in this file.
public boolean	hasNext() Implements java.util.Iterator.hasNext. True if more entries remain in this file.
pack-priv boolean	hasNextBlock()
pack-priv void	initialize(InputStream in, byte[] magic) Initialize the stream by reading from its head.
pack-priv void	initialize(DataFileStream.Header header) Initialize the stream without reading from it.
public Iterator<D>	iterator() Implements java.lang.Iterable.iterator. Returns an iterator over entries in this file.
public D	next() Implements java.util.Iterator.next. Read the next datum in the file.
public D	next(D an instance to reuse. reuse) Read the next datum from the file.
public ByteBuffer	nextBlock() Expert: Return the next block in the file, as binary-encoded data.
pack-priv DataFileStream.DataBlock	nextRawBlock(DataFileStream.DataBlock reuse)
pack-priv byte[]	readMagic()
public void	remove() Overrides default java.util.Iterator.remove. Not supported.
pack-priv Codec	resolveCodec()
pack-priv void	validateMagic(byte[] magic)

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

Field Detail

availableBlock	back to summary
private boolean availableBlock

block	back to summary
private DataFileStream.DataBlock block

blockBuffer	back to summary
pack-priv ByteBuffer blockBuffer

blockCount	back to summary
pack-priv long blockCount

blockRemaining	back to summary
pack-priv long blockRemaining

blockSize	back to summary
private long blockSize

codec	back to summary
private Codec codec

datumIn	back to summary
pack-priv BinaryDecoder datumIn Secondary decoder, for datums. (Different than vin for block segments.)

header	back to summary
private DataFileStream.Header header

reader	back to summary
private DatumReader<D> reader

syncBuffer	back to summary
pack-priv byte[] syncBuffer

vin	back to summary
pack-priv BinaryDecoder vin Decoder on raw input stream. (Used for metadata.)

Constructor Detail

DataFileStream	back to summary
public DataFileStream(InputStream in, DatumReader<D> reader) throws IOException Construct a reader for an input stream. For file-based input, use `DataFileReader`. This will buffer, wrapping with a `java.io.BufferedInputStream` is not necessary.

DataFileStream	back to summary
protected DataFileStream(DatumReader<D> reader) throws IOException create an uninitialized DataFileStream

Method Detail

blockFinished	back to summary
protected void blockFinished() throws IOException

close	back to summary
public void close() throws IOException Implements java.io.Closeable.close. Close this reader. Annotations @Override Exceptions IOException: Doc from java.io.Closeable.close. if an I/O error occurs

back to summary

public void close() throws IOException

Implements java.io.Closeable.close.

Close this reader.

Annotations

@Override

Exceptions

IOException:

Doc from java.io.Closeable.close.

if an I/O error occurs

getBlockCount	back to summary
public long getBlockCount() Expert: Return the count of items in the current block.

getBlockSize	back to summary
public long getBlockSize() Expert: Return the size in bytes (uncompressed) of the current block.

getHeader	back to summary
public DataFileStream.Header getHeader() A handle that can be used to reopen this stream without rereading the head.

getMeta	back to summary
public byte[] getMeta(String key) Return the value of a metadata property.

getMetaKeys	back to summary
public List<String> getMetaKeys() Return the list of keys in the metadata

getMetaLong	back to summary
public long getMetaLong(String key) Return the value of a metadata property.

getMetaString	back to summary
public String getMetaString(String key) Return the value of a metadata property.

getSchema	back to summary
public Schema getSchema() Return the schema used in this file.

hasNext back to summary

hasNext	back to summary
public boolean hasNext() Implements java.util.Iterator.hasNext. True if more entries remain in this file. Returns:boolean Doc from java.util.Iterator.hasNext. `true` if the iteration has more elements Annotations @Override

public boolean hasNext()

Implements java.util.Iterator.hasNext.

True if more entries remain in this file.

Returns:boolean

Doc from java.util.Iterator.hasNext.

true if the iteration has more elements

Annotations

@Override

hasNextBlock	back to summary
pack-priv boolean hasNextBlock()

initialize	back to summary
pack-priv void initialize(InputStream in, byte[] magic) throws IOException Initialize the stream by reading from its head.

initialize	back to summary
pack-priv void initialize(DataFileStream.Header header) throws IOException Initialize the stream without reading from it.

iterator	back to summary
public Iterator<D> iterator() Implements java.lang.Iterable.iterator. Returns an iterator over entries in this file. Note that this iterator is shared with other users of the file: it does not contain a separate pointer into the file. Returns:Iterator<D> Doc from java.lang.Iterable.iterator. an Iterator. Annotations @Override

iterator

back to summary

public Iterator<D> iterator()

Implements java.lang.Iterable.iterator.

Returns an iterator over entries in this file. Note that this iterator is shared with other users of the file: it does not contain a separate pointer into the file.

Returns:Iterator<D>

Doc from java.lang.Iterable.iterator.

an Iterator.

Annotations

@Override

next	back to summary
public D next() Implements java.util.Iterator.next. Read the next datum in the file. Returns:D Doc from java.util.Iterator.next. the next element in the iteration Annotations @Override Exceptions NoSuchElementException: if no more remain in the file.

back to summary

public D next()

Implements java.util.Iterator.next.

Read the next datum in the file.

Returns:D

Doc from java.util.Iterator.next.

the next element in the iteration

Annotations

@Override

Exceptions

NoSuchElementException:: if no more remain in the file.

next	back to summary
public D next(D reuse) throws IOException Read the next datum from the file. Parameters reuse:D an instance to reuse. Exceptions NoSuchElementException: if no more remain in the file.

back to summary

public D next(D reuse) throws IOException

Read the next datum from the file.

Parameters

reuse:D: an instance to reuse.

Exceptions

NoSuchElementException:: if no more remain in the file.

nextBlock	back to summary
public ByteBuffer nextBlock() throws IOException Expert: Return the next block in the file, as binary-encoded data.

nextRawBlock	back to summary
pack-priv DataFileStream.DataBlock nextRawBlock(DataFileStream.DataBlock reuse) throws IOException

readMagic	back to summary
pack-priv byte[] readMagic() throws IOException

remove	back to summary
public void remove() Overrides default java.util.Iterator.remove. Not supported. Annotations @Override

remove

back to summary

public void remove()

Overrides default java.util.Iterator.remove.

Not supported.

Annotations

@Override

resolveCodec	back to summary
pack-priv Codec resolveCodec()

validateMagic	back to summary
pack-priv void validateMagic(byte[] magic) throws InvalidAvroMagicException

org.apache.avro.file back to summary

pack-priv Class DataFileStream.DataBlock

Class Inheritance

extends Object

java.lang.Object
org.apache.avro.file.DataFileStream.DataBlock

Field Summary

Modifier and Type	Field and Description
private int	blockSize
private byte[]	data
private boolean	flushOnWrite
private long	numEntries
private int	offset

Constructor Summary

Access	Constructor and Description
private	DataBlock(long numEntries, int blockSize)
pack-priv	DataBlock(ByteBuffer block, long numEntries)

Method Summary

Modifier and Type	Method and Description
pack-priv void	compressUsing(Codec c)
pack-priv void	decompressUsing(Codec c)
pack-priv ByteBuffer	getAsByteBuffer()
pack-priv int	getBlockSize()
pack-priv byte[]	getData()
pack-priv long	getNumEntries()
pack-priv boolean	isFlushOnWrite()
pack-priv void	setFlushOnWrite(boolean flushOnWrite)
pack-priv void	writeBlockTo(BinaryEncoder e, byte[] sync)

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

Field Detail

blockSize	back to summary
private int blockSize

data	back to summary
private byte[] data

flushOnWrite	back to summary
private boolean flushOnWrite

numEntries	back to summary
private long numEntries

offset	back to summary
private int offset

Constructor Detail

DataBlock	back to summary
private DataBlock(long numEntries, int blockSize)

DataBlock	back to summary
pack-priv DataBlock(ByteBuffer block, long numEntries)

Method Detail

compressUsing	back to summary
pack-priv void compressUsing(Codec c) throws IOException

decompressUsing	back to summary
pack-priv void decompressUsing(Codec c) throws IOException

getAsByteBuffer	back to summary
pack-priv ByteBuffer getAsByteBuffer()

getBlockSize	back to summary
pack-priv int getBlockSize()

getData	back to summary
pack-priv byte[] getData()

getNumEntries	back to summary
pack-priv long getNumEntries()

isFlushOnWrite	back to summary
pack-priv boolean isFlushOnWrite()

setFlushOnWrite	back to summary
pack-priv void setFlushOnWrite(boolean flushOnWrite)

writeBlockTo	back to summary
pack-priv void writeBlockTo(BinaryEncoder e, byte[] sync) throws IOException

public Class DataFileStream<D>

Nested and Inner Type Summary

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail

pack-priv Class DataFileStream.DataBlock

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail

public final Class DataFileStream.Header

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail