Top Description Fields Constructors Methods
com.ning.compress.lzf

public abstract Class ChunkEncoder

extends Object
implements Closeable
Class Inheritance
All Implemented Interfaces
java.io.Closeable, java.lang.AutoCloseable
Known Direct Subclasses
com.ning.compress.lzf.impl.UnsafeChunkEncoder, com.ning.compress.lzf.impl.VanillaChunkEncoder
Imports
java.io.Closeable, .IOException, .OutputStream, com.ning.compress.BufferRecycler
Class that handles actual encoding of individual chunks. Resulting chunks can be compressed or non-compressed; compression is only used if it actually reduces chunk size (including overhead of additional header bytes)

Note that instances are stateful and hence not thread-safe; one instance is meant to be used for processing a sequence of chunks where total length is known.

Author
Tatu Saloranta ([email protected])

Field Summary

Modifier and TypeField and Description
protected byte[]
_encodeBuffer

Buffer in which encoded content is stored during processing

protected final int
_hashModulo

protected int[]
_hashTable

Hash table contains lookup based on 3-byte sequence; key is hash of such triplet, value is offset in buffer.

protected byte[]
_headerBuffer

Small buffer passed to LZFChunk, needed for writing chunk header

protected final BufferRecycler
_recycler

protected static final int
MAX_HASH_SIZE

protected static final int
MAX_OFF

protected static final int
MAX_REF

protected static final int
MIN_BLOCK_TO_COMPRESS

protected static final int
MIN_HASH_SIZE

protected static final int
TAIL_LENGTH

How many tail bytes are we willing to just copy as is, to simplify loop end checks?

Constructor Summary

AccessConstructor and Description
protected
ChunkEncoder(int
Total encoded length; used for calculating size of hash table to use
totalLength)

Uses a ThreadLocal soft-referenced BufferRecycler instance.

protected
ChunkEncoder(int
Total encoded length; used for calculating size of hash table to use
totalLength, BufferRecycler
Buffer recycler instance, for usages where the caller manages the recycler instances
bufferRecycler)

protected
ChunkEncoder(int totalLength, boolean bogus)

Alternate constructor used when we want to avoid allocation encoding buffer, in cases where caller wants full control over allocations.

protected
ChunkEncoder(int totalLength, BufferRecycler bufferRecycler, boolean bogus)

Alternate constructor used when we want to avoid allocation encoding buffer, in cases where caller wants full control over allocations.

Method Summary

Modifier and TypeMethod and Description
public int

Returns:

Offset in output buffer after appending the encoded chunk
appendEncodedChunk(final byte[] input, final int inputPtr, final int inputLen, final byte[] outputBuffer, final int outputPos)

Alternate chunk compression method that will append encoded chunk in pre-allocated buffer.

public int

Returns:

Offset after appending compressed chunk, if compression produces compact enough chunk; otherwise -1 to indicate that no compression resulted.
appendEncodedIfCompresses(final byte[] input, double
Value between 0.05 and 1.10 to indicate maximum relative size of the result to use, in order to append encoded chunk
maxResultRatio, final int inputPtr, final int inputLen, final byte[] outputBuffer, final int outputPos)

Method similar to appendEncodedChunk, but that will only append encoded chunk if it compresses down to specified ratio (also considering header that will be needed); otherwise will return -1 without appending anything.

private static int
calcHashLen(int chunkSize)

public final void
close()

Implements java.io.Closeable.close.

Method to close once encoder is no longer in use.
public void
encodeAndWriteChunk(byte[] data, int offset, int len, OutputStream out)

Method for encoding individual chunk, writing it to given output stream.

public boolean

Returns:

True if compression occurred and chunk was written; false if not.
encodeAndWriteChunkIfCompresses(byte[] data, int offset, int inputLen, OutputStream out, double resultRatio)

Method for encoding individual chunk, writing it to given output stream, if (and only if!) it compresses enough.

public LZFChunk
encodeChunk(byte[] data, int offset, int len)

Method for compressing (or not) individual chunks

public LZFChunk

Returns:

Encoded chunk if (and only if) input compresses down to specified ratio or less; otherwise returns null
encodeChunkIfCompresses(byte[] data, int offset, int inputLen, double
Value between 0.05 and 1.10 to indicate maximum relative size of the result to use, in order to append encoded chunk
maxResultRatio)

Method for compressing individual chunk, if (and only if) it compresses down to specified ratio or less.

public BufferRecycler
getBufferRecycler()

protected final int
hash(int h)

protected abstract int

Returns:

Output pointer after handling content, such that result - originalOutPost is the actual length of compressed chunk (without header)
tryCompress(byte[] in, int inPos, int inEnd, byte[] out, int outPos)

Main workhorse method that will try to compress given chunk, and return end position (offset to byte after last included byte).

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

_encodeBufferback to summary
protected byte[] _encodeBuffer

Buffer in which encoded content is stored during processing

_hashModuloback to summary
protected final int _hashModulo
_hashTableback to summary
protected int[] _hashTable

Hash table contains lookup based on 3-byte sequence; key is hash of such triplet, value is offset in buffer.

_headerBufferback to summary
protected byte[] _headerBuffer

Small buffer passed to LZFChunk, needed for writing chunk header

_recyclerback to summary
protected final BufferRecycler _recycler
MAX_HASH_SIZEback to summary
protected static final int MAX_HASH_SIZE
MAX_OFFback to summary
protected static final int MAX_OFF
MAX_REFback to summary
protected static final int MAX_REF
MIN_BLOCK_TO_COMPRESSback to summary
protected static final int MIN_BLOCK_TO_COMPRESS
MIN_HASH_SIZEback to summary
protected static final int MIN_HASH_SIZE
TAIL_LENGTHback to summary
protected static final int TAIL_LENGTH

How many tail bytes are we willing to just copy as is, to simplify loop end checks? 4 is bare minimum, may be raised to 8?

Constructor Detail

ChunkEncoderback to summary
protected ChunkEncoder(int totalLength)

Uses a ThreadLocal soft-referenced BufferRecycler instance.

Parameters
totalLength:int

Total encoded length; used for calculating size of hash table to use

ChunkEncoderback to summary
protected ChunkEncoder(int totalLength, BufferRecycler bufferRecycler)
Parameters
totalLength:int

Total encoded length; used for calculating size of hash table to use

bufferRecycler:BufferRecycler

Buffer recycler instance, for usages where the caller manages the recycler instances

ChunkEncoderback to summary
protected ChunkEncoder(int totalLength, boolean bogus)

Alternate constructor used when we want to avoid allocation encoding buffer, in cases where caller wants full control over allocations.

ChunkEncoderback to summary
protected ChunkEncoder(int totalLength, BufferRecycler bufferRecycler, boolean bogus)

Alternate constructor used when we want to avoid allocation encoding buffer, in cases where caller wants full control over allocations.

Method Detail

appendEncodedChunkback to summary
public int appendEncodedChunk(final byte[] input, final int inputPtr, final int inputLen, final byte[] outputBuffer, final int outputPos)

Alternate chunk compression method that will append encoded chunk in pre-allocated buffer. Note that caller must ensure that the buffer is large enough to hold not just encoded result but also intermediate result; latter may be up to 4% larger than input; caller may use LZFEncoder#estimateMaxWorkspaceSize(int) to calculate necessary buffer size.

Returns:int

Offset in output buffer after appending the encoded chunk

appendEncodedIfCompressesback to summary
public int appendEncodedIfCompresses(final byte[] input, double maxResultRatio, final int inputPtr, final int inputLen, final byte[] outputBuffer, final int outputPos)

Method similar to appendEncodedChunk, but that will only append encoded chunk if it compresses down to specified ratio (also considering header that will be needed); otherwise will return -1 without appending anything.

Parameters
maxResultRatio:double

Value between 0.05 and 1.10 to indicate maximum relative size of the result to use, in order to append encoded chunk

Returns:int

Offset after appending compressed chunk, if compression produces compact enough chunk; otherwise -1 to indicate that no compression resulted.

calcHashLenback to summary
private static int calcHashLen(int chunkSize)
closeback to summary
public final void close()

Implements java.io.Closeable.close.

Method to close once encoder is no longer in use.

Note

after calling this method, further calls to encodeChunk will fail

Annotations
@Override
encodeAndWriteChunkback to summary
public void encodeAndWriteChunk(byte[] data, int offset, int len, OutputStream out) throws IOException

Method for encoding individual chunk, writing it to given output stream.

encodeAndWriteChunkIfCompressesback to summary
public boolean encodeAndWriteChunkIfCompresses(byte[] data, int offset, int inputLen, OutputStream out, double resultRatio) throws IOException

Method for encoding individual chunk, writing it to given output stream, if (and only if!) it compresses enough.

Returns:boolean

True if compression occurred and chunk was written; false if not.

encodeChunkback to summary
public LZFChunk encodeChunk(byte[] data, int offset, int len)

Method for compressing (or not) individual chunks

encodeChunkIfCompressesback to summary
public LZFChunk encodeChunkIfCompresses(byte[] data, int offset, int inputLen, double maxResultRatio)

Method for compressing individual chunk, if (and only if) it compresses down to specified ratio or less.

Parameters
maxResultRatio:double

Value between 0.05 and 1.10 to indicate maximum relative size of the result to use, in order to append encoded chunk

Returns:LZFChunk

Encoded chunk if (and only if) input compresses down to specified ratio or less; otherwise returns null

getBufferRecyclerback to summary
public BufferRecycler getBufferRecycler()
hashback to summary
protected final int hash(int h)
tryCompressback to summary
protected abstract int tryCompress(byte[] in, int inPos, int inEnd, byte[] out, int outPos)

Main workhorse method that will try to compress given chunk, and return end position (offset to byte after last included byte). Result will be "raw" encoded contents without chunk header information: caller is responsible for prepending header, if it chooses to use encoded data; it may also choose to instead create an uncompressed chunk.

Returns:int

Output pointer after handling content, such that result - originalOutPost is the actual length of compressed chunk (without header)