Top Description Fields Constructors Methods
org.apache.commons.compress.compressors.snappy

public Class SnappyCompressorOutputStream

extends CompressorOutputStream
Class Inheritance
Imports
java.io.IOException, .OutputStream, org.apache.commons.compress.compressors.CompressorOutputStream, org.apache.commons.compress.compressors.lz77support.LZ77Compressor, .Parameters, org.apache.commons.compress.utils.ByteUtils
CompressorOutputStream for the raw Snappy format.

This implementation uses an internal buffer in order to handle the back-references that are at the heart of the LZ77 algorithm. The size of the buffer must be at least as big as the biggest offset used in the compressed stream. The current version of the Snappy algorithm as defined by Google works on 32k blocks and doesn't contain offsets bigger than 32k which is the default block size used by this class.

The raw Snappy format requires the uncompressed size to be written at the beginning of the stream using a varint representation, i.e. the number of bytes needed to write the information is not known before the uncompressed size is known. We've chosen to make the uncompressedSize a parameter of the constructor in favor of buffering the whole output until the size is known. When using the FramedSnappyCompressorOutputStream this limitation is taken care of by the warpping framing format.

Since
1.14
See Also
Snappy compressed format description

Field Summary

Modifier and TypeField and Description
private final LZ77Compressor
compressor

private final ByteUtils.ByteConsumer
consumer

private boolean
finished

private static final int
FOUR_BYTE_COPY_TAG

private static final int
FOUR_SIZE_BYTE_MARKER

private static final int
MAX_LITERAL_SIZE_WITH_ONE_SIZE_BYTE

private static final int
MAX_LITERAL_SIZE_WITH_THREE_SIZE_BYTES

private static final int
MAX_LITERAL_SIZE_WITH_TWO_SIZE_BYTES

private static final int
MAX_LITERAL_SIZE_WITHOUT_SIZE_BYTES

private static final int
MAX_MATCH_LENGTH

private static final int
MAX_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE

private static final int
MAX_OFFSET_WITH_ONE_OFFSET_BYTE

private static final int
MAX_OFFSET_WITH_TWO_OFFSET_BYTES

private static final int
MIN_MATCH_LENGTH

private static final int
MIN_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE

private static final int
ONE_BYTE_COPY_TAG

private static final int
ONE_SIZE_BYTE_MARKER

private final byte[]
oneByte

private final OutputStream
os

private static final int
THREE_SIZE_BYTE_MARKER

private static final int
TWO_BYTE_COPY_TAG

private static final int
TWO_SIZE_BYTE_MARKER

Constructor Summary

AccessConstructor and Description
public
SnappyCompressorOutputStream(final OutputStream
the outputstream to write compressed data to
os, final long
the uncompressed size of data
uncompressedSize)

Constructor using the default block size of 32k.

public
SnappyCompressorOutputStream(final OutputStream
the outputstream to write compressed data to
os, final long
the uncompressed size of data
uncompressedSize, final int
the block size used - must be a power of two
blockSize)

Constructor using a configurable block size.

public
SnappyCompressorOutputStream(final OutputStream
the outputstream to write compressed data to
os, final long
the uncompressed size of data
uncompressedSize, final Parameters
the parameters to use by the compressor - note that the format itself imposes some limits like a maximum match length of 64 bytes
params)

Constructor providing full control over the underlying LZ77 compressor.

Method Summary

Modifier and TypeMethod and Description
public void
close()

Overrides java.io.OutputStream.close.

Implements java.io.Closeable.close.

Closes this output stream and releases any system resources associated with this stream.
public static Parameters.Builder

Returns:

a builder correctly configured for the Snappy algorithm using the gven block size
createParameterBuilder(final int
the block size.
blockSize)

Returns a builder correctly configured for the Snappy algorithm using the gven block size.

public void
finish()

Compresses all remaining data and writes it to the stream, doesn't close the underlying stream.

public void
write(final byte[]
the data.
data, final int
the start offset in the data.
off, final int
the number of bytes to write.
len)

Overrides java.io.OutputStream.write.

Writes len bytes from the specified byte array starting at offset off to this output stream.
public void
write(final int
the byte.
b)

Implements abstract java.io.OutputStream.write.

Writes the specified byte to this output stream.
private void
writeBackReference(final LZ77Compressor.BackReference block)

private void
writeBackReferenceWithFourOffsetBytes(final int len, final int offset)

private void
writeBackReferenceWithLittleEndianOffset(final int tag, final int offsetBytes, final int len, final int offset)

private void
writeBackReferenceWithOneOffsetByte(final int len, final int offset)

private void
writeBackReferenceWithTwoOffsetBytes(final int len, final int offset)

private void
writeLiteralBlock(final LZ77Compressor.LiteralBlock block)

private void
writeLiteralBlockFourSizeBytes(final LZ77Compressor.LiteralBlock block, final int len)

private void
writeLiteralBlockNoSizeBytes(final LZ77Compressor.LiteralBlock block, final int len)

private void
writeLiteralBlockOneSizeByte(final LZ77Compressor.LiteralBlock block, final int len)

private void
writeLiteralBlockThreeSizeBytes(final LZ77Compressor.LiteralBlock block, final int len)

private void
writeLiteralBlockTwoSizeBytes(final LZ77Compressor.LiteralBlock block, final int len)

private void
writeLiteralBlockWithSize(final int tagByte, final int sizeBytes, final int len, final LZ77Compressor.LiteralBlock block)

private void
writeLittleEndian(final int numBytes, final int num)

private void
writeUncompressedSize(long uncompressedSize)

Field Detail

compressorback to summary
private final LZ77Compressor compressor
consumerback to summary
private final ByteUtils.ByteConsumer consumer
finishedback to summary
private boolean finished
FOUR_BYTE_COPY_TAGback to summary
private static final int FOUR_BYTE_COPY_TAG
FOUR_SIZE_BYTE_MARKERback to summary
private static final int FOUR_SIZE_BYTE_MARKER
MAX_LITERAL_SIZE_WITH_ONE_SIZE_BYTEback to summary
private static final int MAX_LITERAL_SIZE_WITH_ONE_SIZE_BYTE
MAX_LITERAL_SIZE_WITH_THREE_SIZE_BYTESback to summary
private static final int MAX_LITERAL_SIZE_WITH_THREE_SIZE_BYTES
MAX_LITERAL_SIZE_WITH_TWO_SIZE_BYTESback to summary
private static final int MAX_LITERAL_SIZE_WITH_TWO_SIZE_BYTES
MAX_LITERAL_SIZE_WITHOUT_SIZE_BYTESback to summary
private static final int MAX_LITERAL_SIZE_WITHOUT_SIZE_BYTES
MAX_MATCH_LENGTHback to summary
private static final int MAX_MATCH_LENGTH
MAX_MATCH_LENGTH_WITH_ONE_OFFSET_BYTEback to summary
private static final int MAX_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE
MAX_OFFSET_WITH_ONE_OFFSET_BYTEback to summary
private static final int MAX_OFFSET_WITH_ONE_OFFSET_BYTE
MAX_OFFSET_WITH_TWO_OFFSET_BYTESback to summary
private static final int MAX_OFFSET_WITH_TWO_OFFSET_BYTES
MIN_MATCH_LENGTHback to summary
private static final int MIN_MATCH_LENGTH
MIN_MATCH_LENGTH_WITH_ONE_OFFSET_BYTEback to summary
private static final int MIN_MATCH_LENGTH_WITH_ONE_OFFSET_BYTE
ONE_BYTE_COPY_TAGback to summary
private static final int ONE_BYTE_COPY_TAG
ONE_SIZE_BYTE_MARKERback to summary
private static final int ONE_SIZE_BYTE_MARKER
oneByteback to summary
private final byte[] oneByte
osback to summary
private final OutputStream os
THREE_SIZE_BYTE_MARKERback to summary
private static final int THREE_SIZE_BYTE_MARKER
TWO_BYTE_COPY_TAGback to summary
private static final int TWO_BYTE_COPY_TAG
TWO_SIZE_BYTE_MARKERback to summary
private static final int TWO_SIZE_BYTE_MARKER

Constructor Detail

SnappyCompressorOutputStreamback to summary
public SnappyCompressorOutputStream(final OutputStream os, final long uncompressedSize) throws IOException

Constructor using the default block size of 32k.

Parameters
os:OutputStream

the outputstream to write compressed data to

uncompressedSize:long

the uncompressed size of data

Exceptions
IOException:
if writing of the size fails
SnappyCompressorOutputStreamback to summary
public SnappyCompressorOutputStream(final OutputStream os, final long uncompressedSize, final int blockSize) throws IOException

Constructor using a configurable block size.

Parameters
os:OutputStream

the outputstream to write compressed data to

uncompressedSize:long

the uncompressed size of data

blockSize:int

the block size used - must be a power of two

Exceptions
IOException:
if writing of the size fails
SnappyCompressorOutputStreamback to summary
public SnappyCompressorOutputStream(final OutputStream os, final long uncompressedSize, final Parameters params) throws IOException

Constructor providing full control over the underlying LZ77 compressor.

Parameters
os:OutputStream

the outputstream to write compressed data to

uncompressedSize:long

the uncompressed size of data

params:Parameters

the parameters to use by the compressor - note that the format itself imposes some limits like a maximum match length of 64 bytes

Exceptions
IOException:
if writing of the size fails

Method Detail

closeback to summary
public void close() throws IOException

Overrides java.io.OutputStream.close.

Implements java.io.Closeable.close.

Doc from java.io.OutputStream.close.

Closes this output stream and releases any system resources associated with this stream. The general contract of close is that it closes the output stream. A closed stream cannot perform output operations and cannot be reopened.

Annotations
@Override
Exceptions
IOException:
if an I/O error occurs.
createParameterBuilderback to summary
public static Parameters.Builder createParameterBuilder(final int blockSize)

Returns a builder correctly configured for the Snappy algorithm using the gven block size.

Parameters
blockSize:int

the block size.

Returns:Parameters.Builder

a builder correctly configured for the Snappy algorithm using the gven block size

finishback to summary
public void finish() throws IOException

Compresses all remaining data and writes it to the stream, doesn't close the underlying stream.

Exceptions
IOException:
if an error occurs
writeback to summary
public void write(final byte[] data, final int off, final int len) throws IOException

Overrides java.io.OutputStream.write.

Doc from java.io.OutputStream.write.

Writes len bytes from the specified byte array starting at offset off to this output stream. The general contract for write(b, off, len) is that some of the bytes in the array b are written to the output stream in order; element b[off] is the first byte written and b[off+len-1] is the last byte written by this operation.

If b is null, a NullPointerException is thrown.

If off is negative, or len is negative, or off+len is greater than the length of the array b, then an IndexOutOfBoundsException is thrown.

Parameters
data:byte[]

the data.

off:int

the start offset in the data.

len:int

the number of bytes to write.

Annotations
@Override
Exceptions
IOException:
if an I/O error occurs. In particular, an IOException is thrown if the output stream is closed.
writeback to summary
public void write(final int b) throws IOException

Implements abstract java.io.OutputStream.write.

Doc from java.io.OutputStream.write.

Writes the specified byte to this output stream. The general contract for write is that one byte is written to the output stream. The byte to be written is the eight low-order bits of the argument b. The 24 high-order bits of b are ignored.

Parameters
b:int

the byte.

Annotations
@Override
Exceptions
IOException:
if an I/O error occurs. In particular, an IOException may be thrown if the output stream has been closed.
writeBackReferenceback to summary
private void writeBackReference(final LZ77Compressor.BackReference block) throws IOException
writeBackReferenceWithFourOffsetBytesback to summary
private void writeBackReferenceWithFourOffsetBytes(final int len, final int offset) throws IOException
writeBackReferenceWithLittleEndianOffsetback to summary
private void writeBackReferenceWithLittleEndianOffset(final int tag, final int offsetBytes, final int len, final int offset) throws IOException
writeBackReferenceWithOneOffsetByteback to summary
private void writeBackReferenceWithOneOffsetByte(final int len, final int offset) throws IOException
writeBackReferenceWithTwoOffsetBytesback to summary
private void writeBackReferenceWithTwoOffsetBytes(final int len, final int offset) throws IOException
writeLiteralBlockback to summary
private void writeLiteralBlock(final LZ77Compressor.LiteralBlock block) throws IOException
writeLiteralBlockFourSizeBytesback to summary
private void writeLiteralBlockFourSizeBytes(final LZ77Compressor.LiteralBlock block, final int len) throws IOException
writeLiteralBlockNoSizeBytesback to summary
private void writeLiteralBlockNoSizeBytes(final LZ77Compressor.LiteralBlock block, final int len) throws IOException
writeLiteralBlockOneSizeByteback to summary
private void writeLiteralBlockOneSizeByte(final LZ77Compressor.LiteralBlock block, final int len) throws IOException
writeLiteralBlockThreeSizeBytesback to summary
private void writeLiteralBlockThreeSizeBytes(final LZ77Compressor.LiteralBlock block, final int len) throws IOException
writeLiteralBlockTwoSizeBytesback to summary
private void writeLiteralBlockTwoSizeBytes(final LZ77Compressor.LiteralBlock block, final int len) throws IOException
writeLiteralBlockWithSizeback to summary
private void writeLiteralBlockWithSize(final int tagByte, final int sizeBytes, final int len, final LZ77Compressor.LiteralBlock block) throws IOException
writeLittleEndianback to summary
private void writeLittleEndian(final int numBytes, final int num) throws IOException
writeUncompressedSizeback to summary
private void writeUncompressedSize(long uncompressedSize) throws IOException