Top Description Inners Fields Constructors Methods
io.netty.handler.codec.compression

public final Class Snappy

extends Object
Class Inheritance
Imports
io.netty.buffer.ByteBuf, io.netty.util.internal.MathUtil
Uncompresses an input ByteBuf encoded with Snappy compression into an output ByteBuf. See snappy format.

Nested and Inner Type Summary

Modifier and TypeClass and Description
private static enum
Snappy.State

Field Summary

Modifier and TypeField and Description
private static final int
COPY_1_BYTE_OFFSET

private static final int
COPY_2_BYTE_OFFSET

private static final int
COPY_4_BYTE_OFFSET

private static final int
LITERAL

private static final int
MAX_HT_SIZE

private static final int
MIN_COMPRESSIBLE_BYTES

private static final int
NOT_ENOUGH_INPUT

private static final int
PREAMBLE_NOT_FULL

private Snappy.State
state

private byte
tag

private int
written

Constructor Summary

AccessConstructor and Description
public
Snappy()

Method Summary

Modifier and TypeMethod and Description
private static int

Returns:

The minimum number of bits required to encode the supplied value
bitsToEncode(int
The value to calculate the minimum number of bits required to encode
value)

Calculates the minimum number of bits required to encode a value.

pack-priv static int
calculateChecksum(ByteBuf
The input data to calculate the CRC32C checksum of
data)

Computes the CRC32C checksum of the supplied data and performs the "mask" operation on the computed checksum

pack-priv static int
calculateChecksum(ByteBuf
The input data to calculate the CRC32C checksum of
data, int offset, int length)

Computes the CRC32C checksum of the supplied data and performs the "mask" operation on the computed checksum

public void
decode(ByteBuf in, ByteBuf out)

private static int

Returns:

The number of bytes appended to the output buffer, or -1 to indicate "try again later"
decodeCopyWith1ByteOffset(byte
The tag used to identify this as a copy is also used to encode the length and part of the offset
tag, ByteBuf
The input buffer to read from
in, ByteBuf
The output buffer to write to
out, int writtenSoFar)

Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.

private static int

Returns:

The number of bytes appended to the output buffer, or -1 to indicate "try again later"
decodeCopyWith2ByteOffset(byte
The tag used to identify this as a copy is also used to encode the length and part of the offset
tag, ByteBuf
The input buffer to read from
in, ByteBuf
The output buffer to write to
out, int writtenSoFar)

Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.

private static int

Returns:

The number of bytes appended to the output buffer, or -1 to indicate "try again later"
decodeCopyWith4ByteOffset(byte
The tag used to identify this as a copy is also used to encode the length and part of the offset
tag, ByteBuf
The input buffer to read from
in, ByteBuf
The output buffer to write to
out, int writtenSoFar)

Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.

pack-priv static int

Returns:

The number of bytes appended to the output buffer, or -1 to indicate "try again later"
decodeLiteral(byte
The tag that identified this segment as a literal is also used to encode part of the length of the data
tag, ByteBuf
The input buffer to read the literal from
in, ByteBuf
The output buffer to write the literal to
out)

Reads a literal from the input buffer directly to the output buffer.

public void
encode(final ByteBuf in, final ByteBuf out, final int length)

private static void
encodeCopy(ByteBuf
The output buffer to write the copy pointer to
out, int
The offset at which the original instance lies
offset, int
The length of the original instance
length)

Encodes a series of copies, each at most 64 bytes in length.

private static void
encodeCopyWithOffset(ByteBuf out, int offset, int length)

pack-priv static void
encodeLiteral(ByteBuf
The input buffer to copy from
in, ByteBuf
The output buffer to copy to
out, int
The length of the literal to copy
length)

Writes a literal to the supplied output buffer by directly copying from the input buffer.

private static int

Returns:

The number of bytes for which our candidate copy is a repeat of
findMatchingLength(ByteBuf
The input buffer to scan over
in, int
The index in the input buffer to start scanning from
minIndex, int
The index of the start of our copy
inIndex, int
The length of our input buffer
maxIndex)

Iterates over the supplied input buffer between the supplied minIndex and maxIndex to find how long our matched copy overlaps with an already-written literal value.

private static short[]

Returns:

An appropriately sized empty hashtable
getHashTable(int
The size of our input, ie. the number of bytes we need to encode
inputSize)

Creates an appropriately sized hashtable for the given input size

pack-priv int

Returns:

The calculated length based on the input buffer, or 0 if no preamble is able to be calculated
getPreamble(ByteBuf
The input buffer to get the preamble from
in)

Get the length varint (a series of bytes, where the lower 7 bits are data and the upper bit is a flag to indicate more bytes to be read).

private static int

Returns:

A 32-bit hash of 4 bytes located at index
hash(ByteBuf
The input buffer to read 4 bytes from
in, int
The index to read at
index, int
The shift value, for ensuring that the resulting value is within the range of our hash table size
shift)

Hashes the 4 bytes located at index, shifting the resulting hash into the appropriate range for our hash table.

pack-priv static int

Returns:

The masked checksum
maskChecksum(long
The actual checksum of the data
checksum)

From the spec: "Checksums are not stored directly, but masked, as checksumming data and then its own checksum can be problematic.

private static int

Returns:

The calculated length based on the input buffer, or 0 if no preamble is able to be calculated
readPreamble(ByteBuf
The input buffer to read the preamble from
in)

Reads the length varint (a series of bytes, where the lower 7 bits are data and the upper bit is a flag to indicate more bytes to be read).

public void
reset()

pack-priv static void
validateChecksum(int
The checksum decoded from the stream to compare against
expectedChecksum, ByteBuf
The input data to calculate the CRC32C checksum of
data)

Computes the CRC32C checksum of the supplied data, performs the "mask" operation on the computed checksum, and then compares the resulting masked checksum to the supplied checksum.

pack-priv static void
validateChecksum(int
The checksum decoded from the stream to compare against
expectedChecksum, ByteBuf
The input data to calculate the CRC32C checksum of
data, int offset, int length)

Computes the CRC32C checksum of the supplied data, performs the "mask" operation on the computed checksum, and then compares the resulting masked checksum to the supplied checksum.

private static void
validateOffset(int
The offset extracted from the compressed reference
offset, int
The number of bytes read so far from this chunk
chunkSizeSoFar)

Validates that the offset extracted from a compressed reference is within the permissible bounds of an offset (0 < offset < Integer.MAX_VALUE), and does not exceed the length of the chunk currently read so far.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

COPY_1_BYTE_OFFSETback to summary
private static final int COPY_1_BYTE_OFFSET
COPY_2_BYTE_OFFSETback to summary
private static final int COPY_2_BYTE_OFFSET
COPY_4_BYTE_OFFSETback to summary
private static final int COPY_4_BYTE_OFFSET
LITERALback to summary
private static final int LITERAL
MAX_HT_SIZEback to summary
private static final int MAX_HT_SIZE
MIN_COMPRESSIBLE_BYTESback to summary
private static final int MIN_COMPRESSIBLE_BYTES
NOT_ENOUGH_INPUTback to summary
private static final int NOT_ENOUGH_INPUT
PREAMBLE_NOT_FULLback to summary
private static final int PREAMBLE_NOT_FULL
stateback to summary
private Snappy.State state
tagback to summary
private byte tag
writtenback to summary
private int written

Constructor Detail

Snappyback to summary
public Snappy()

Method Detail

bitsToEncodeback to summary
private static int bitsToEncode(int value)

Calculates the minimum number of bits required to encode a value. This can then in turn be used to calculate the number of septets or octets (as appropriate) to use to encode a length parameter.

Parameters
value:int

The value to calculate the minimum number of bits required to encode

Returns:int

The minimum number of bits required to encode the supplied value

calculateChecksumback to summary
pack-priv static int calculateChecksum(ByteBuf data)

Computes the CRC32C checksum of the supplied data and performs the "mask" operation on the computed checksum

Parameters
data:ByteBuf

The input data to calculate the CRC32C checksum of

calculateChecksumback to summary
pack-priv static int calculateChecksum(ByteBuf data, int offset, int length)

Computes the CRC32C checksum of the supplied data and performs the "mask" operation on the computed checksum

Parameters
data:ByteBuf

The input data to calculate the CRC32C checksum of

decodeback to summary
public void decode(ByteBuf in, ByteBuf out)
decodeCopyWith1ByteOffsetback to summary
private static int decodeCopyWith1ByteOffset(byte tag, ByteBuf in, ByteBuf out, int writtenSoFar)

Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.

Parameters
tag:byte

The tag used to identify this as a copy is also used to encode the length and part of the offset

in:ByteBuf

The input buffer to read from

out:ByteBuf

The output buffer to write to

Returns:int

The number of bytes appended to the output buffer, or -1 to indicate "try again later"

Exceptions
DecompressionException:
If the read offset is invalid
decodeCopyWith2ByteOffsetback to summary
private static int decodeCopyWith2ByteOffset(byte tag, ByteBuf in, ByteBuf out, int writtenSoFar)

Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.

Parameters
tag:byte

The tag used to identify this as a copy is also used to encode the length and part of the offset

in:ByteBuf

The input buffer to read from

out:ByteBuf

The output buffer to write to

Returns:int

The number of bytes appended to the output buffer, or -1 to indicate "try again later"

Exceptions
DecompressionException:
If the read offset is invalid
decodeCopyWith4ByteOffsetback to summary
private static int decodeCopyWith4ByteOffset(byte tag, ByteBuf in, ByteBuf out, int writtenSoFar)

Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.

Parameters
tag:byte

The tag used to identify this as a copy is also used to encode the length and part of the offset

in:ByteBuf

The input buffer to read from

out:ByteBuf

The output buffer to write to

Returns:int

The number of bytes appended to the output buffer, or -1 to indicate "try again later"

Exceptions
DecompressionException:
If the read offset is invalid
decodeLiteralback to summary
pack-priv static int decodeLiteral(byte tag, ByteBuf in, ByteBuf out)

Reads a literal from the input buffer directly to the output buffer. A "literal" is an uncompressed segment of data stored directly in the byte stream.

Parameters
tag:byte

The tag that identified this segment as a literal is also used to encode part of the length of the data

in:ByteBuf

The input buffer to read the literal from

out:ByteBuf

The output buffer to write the literal to

Returns:int

The number of bytes appended to the output buffer, or -1 to indicate "try again later"

encodeback to summary
public void encode(final ByteBuf in, final ByteBuf out, final int length)
encodeCopyback to summary
private static void encodeCopy(ByteBuf out, int offset, int length)

Encodes a series of copies, each at most 64 bytes in length.

Parameters
out:ByteBuf

The output buffer to write the copy pointer to

offset:int

The offset at which the original instance lies

length:int

The length of the original instance

encodeCopyWithOffsetback to summary
private static void encodeCopyWithOffset(ByteBuf out, int offset, int length)
encodeLiteralback to summary
pack-priv static void encodeLiteral(ByteBuf in, ByteBuf out, int length)

Writes a literal to the supplied output buffer by directly copying from the input buffer. The literal is taken from the current readerIndex up to the supplied length.

Parameters
in:ByteBuf

The input buffer to copy from

out:ByteBuf

The output buffer to copy to

length:int

The length of the literal to copy

findMatchingLengthback to summary
private static int findMatchingLength(ByteBuf in, int minIndex, int inIndex, int maxIndex)

Iterates over the supplied input buffer between the supplied minIndex and maxIndex to find how long our matched copy overlaps with an already-written literal value.

Parameters
in:ByteBuf

The input buffer to scan over

minIndex:int

The index in the input buffer to start scanning from

inIndex:int

The index of the start of our copy

maxIndex:int

The length of our input buffer

Returns:int

The number of bytes for which our candidate copy is a repeat of

getHashTableback to summary
private static short[] getHashTable(int inputSize)

Creates an appropriately sized hashtable for the given input size

Parameters
inputSize:int

The size of our input, ie. the number of bytes we need to encode

Returns:short[]

An appropriately sized empty hashtable

getPreambleback to summary
pack-priv int getPreamble(ByteBuf in)

Get the length varint (a series of bytes, where the lower 7 bits are data and the upper bit is a flag to indicate more bytes to be read).

Parameters
in:ByteBuf

The input buffer to get the preamble from

Returns:int

The calculated length based on the input buffer, or 0 if no preamble is able to be calculated

hashback to summary
private static int hash(ByteBuf in, int index, int shift)

Hashes the 4 bytes located at index, shifting the resulting hash into the appropriate range for our hash table.

Parameters
in:ByteBuf

The input buffer to read 4 bytes from

index:int

The index to read at

shift:int

The shift value, for ensuring that the resulting value is within the range of our hash table size

Returns:int

A 32-bit hash of 4 bytes located at index

maskChecksumback to summary
pack-priv static int maskChecksum(long checksum)

From the spec: "Checksums are not stored directly, but masked, as checksumming data and then its own checksum can be problematic. The masking is the same as used in Apache Hadoop: Rotate the checksum by 15 bits, then add the constant 0xa282ead8 (using wraparound as normal for unsigned integers)."

Parameters
checksum:long

The actual checksum of the data

Returns:int

The masked checksum

readPreambleback to summary
private static int readPreamble(ByteBuf in)

Reads the length varint (a series of bytes, where the lower 7 bits are data and the upper bit is a flag to indicate more bytes to be read).

Parameters
in:ByteBuf

The input buffer to read the preamble from

Returns:int

The calculated length based on the input buffer, or 0 if no preamble is able to be calculated

resetback to summary
public void reset()
validateChecksumback to summary
pack-priv static void validateChecksum(int expectedChecksum, ByteBuf data)

Computes the CRC32C checksum of the supplied data, performs the "mask" operation on the computed checksum, and then compares the resulting masked checksum to the supplied checksum.

Parameters
expectedChecksum:int

The checksum decoded from the stream to compare against

data:ByteBuf

The input data to calculate the CRC32C checksum of

Exceptions
DecompressionException:
If the calculated and supplied checksums do not match
validateChecksumback to summary
pack-priv static void validateChecksum(int expectedChecksum, ByteBuf data, int offset, int length)

Computes the CRC32C checksum of the supplied data, performs the "mask" operation on the computed checksum, and then compares the resulting masked checksum to the supplied checksum.

Parameters
expectedChecksum:int

The checksum decoded from the stream to compare against

data:ByteBuf

The input data to calculate the CRC32C checksum of

Exceptions
DecompressionException:
If the calculated and supplied checksums do not match
validateOffsetback to summary
private static void validateOffset(int offset, int chunkSizeSoFar)

Validates that the offset extracted from a compressed reference is within the permissible bounds of an offset (0 < offset < Integer.MAX_VALUE), and does not exceed the length of the chunk currently read so far.

Parameters
offset:int

The offset extracted from the compressed reference

chunkSizeSoFar:int

The number of bytes read so far from this chunk

Exceptions
DecompressionException:
if the offset is invalid
io.netty.handler.codec.compression back to summary

private final Enum Snappy.State

extends Enum<Snappy.State>
Class Inheritance

Field Summary

Modifier and TypeField and Description
public static final Snappy.State
READING_COPY

public static final Snappy.State
READING_LITERAL

public static final Snappy.State
READING_PREAMBLE

public static final Snappy.State
READING_TAG

Constructor Summary

AccessConstructor and Description
private
State()

Method Summary

Modifier and TypeMethod and Description
public static Snappy.State
valueOf(String name)

public static Snappy.State[]
values()

Inherited from java.lang.Enum:
clonecompareTodescribeConstableequalsfinalizegetDeclaringClasshashCodenameordinaltoStringvalueOf

Field Detail

READING_COPYback to summary
public static final Snappy.State READING_COPY
READING_LITERALback to summary
public static final Snappy.State READING_LITERAL
READING_PREAMBLEback to summary
public static final Snappy.State READING_PREAMBLE
READING_TAGback to summary
public static final Snappy.State READING_TAG

Constructor Detail

Stateback to summary
private State()

Method Detail

valueOfback to summary
public static Snappy.State valueOf(String name)
valuesback to summary
public static Snappy.State[] values()