Top Description Inners Fields Constructors Methods
com.amazon.ion.impl

public Class ResizingPipedInputStream

extends InputStream
Class Inheritance
Imports
java.io.EOFException, .IOException, .InputStream, .OutputStream, java.nio.BufferOverflowException, .ByteBuffer

Manages a resizing buffer for production and consumption of data within a single thread. Buffered bytes may be consumed through the InputStream interface. This provides a few benefits over using a PipedOutputStream/PipedInputStream pair in a single thread:
  1. There is no risk of deadlock. Piped streams, which are intended for producing data in one thread and consuming it in another, will block on read when no data is available and block on write when the buffer is full. In a single-threaded context, avoiding deadlock on read requires checking that bytes are available before every read. Avoiding deadlock on write would require checking that the buffer is not full before every write, but there is no built-in, publicly-accessible way of doing this with a PipedInputStream/PipedOutputStream.
  2. The buffer can grow. Piped streams use a fixed-size buffer that causes blocking when full. If used in a single-thread, this serves as a hard limit on the amount of data that can be written without a matching read. This can require arbitrary limits on data size to be imposed by the application. The ResizingPipedInputStream imposes no such limitation, but optionally allows for a maximum buffer size to be configured to protect against unbounded growth.

Nested and Inner Type Summary

Modifier and TypeClass and Description
pack-priv static interface
ResizingPipedInputStream.NotificationConsumer

Handler of notifications provided by the ResizingPipedInputStream.

Field Summary

Modifier and TypeField and Description
private int
private int
private byte[]
buffer

The raw buffer.

private ByteBuffer
byteBuffer

View to the raw buffer.

private int
private final int
initialBufferSize

The initial size of the buffer and the number of bytes by which the size of the buffer will increase each time it grows, unless it must grow by a smaller amount to fit within 'maximumBufferSize'.

private final int
maximumBufferSize

The maximum size of the buffer.

private static final ResizingPipedInputStream.NotificationConsumer
NO_OP_NOTIFICATION_CONSUMER

A NotificationConsumer that does nothing.

private ResizingPipedInputStream.NotificationConsumer
notificationConsumer

The NotificationConsumer currently registered.

private int
readIndex

The index of the next byte in the buffer that is available to be read.

private static final int
SINGLE_BYTE_MASK

Mask to isolate a single byte.

private int
private final boolean
useBoundary

Whether to use a boundary to limit the number of available bytes.

private int
writeIndex

The index at which the next byte received will be written.

Constructor Summary

AccessConstructor and Description
public
ResizingPipedInputStream(final int
the initial size of the buffer. When full, the buffer will grow by this many bytes. The buffer always stores bytes contiguously, so growth requires allocation of a new buffer capable of holding the new capacity and copying of the existing bytes into the new buffer. As such, a size should be chosen carefully such that growth is expected to occur rarely, if ever.
initialBufferSize
)

Constructor.

pack-priv
ResizingPipedInputStream(final int
the initial size of the buffer. When full, the buffer will grow by this many bytes. The buffer always stores bytes contiguously, so growth requires allocation of a new buffer capable of holding the new capacity and copying of the existing bytes into the new buffer. As such, a size should be chosen carefully such that growth is expected to occur rarely, if ever.
initialBufferSize
,
final int
the maximum size of the buffer. If a call to `receive` attempts to transfer an amount of bytes that would cause the buffer to exceed this size, a BufferOverflowException will be thrown. Must be greater than or equal to the initial buffer size.
maximumBufferSize
,
final boolean
whether to use a boundary to limit the number of available bytes. This can be used to buffer arbitrarily-sized chunks of bytes without making them available for consumption. When true, the boundary must be manually extended (see extendBoundary(int) to make these bytes available. When false, all buffered bytes will be available to read.
useBoundary
)

Constructor.

Method Summary

Modifier and TypeMethod and Description
public int

Returns:

the exact number of bytes available in the buffer.
available
()

Overrides java.io.InputStream.available.

Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking, which may be 0, or 0 when end of stream is detected.

pack-priv int

Returns:

the number of bytes buffered beyond the boundary. This is equivalent to subtracting available() from size().
availableBeyondBoundary
()

public int

Returns:

the capacity of the buffer, which is always less than or equal to 'maximumBufferSize'.
capacity
()

pack-priv void
clear()

Clears the buffer.

pack-priv void
consolidate(int
the position to move bytes from. Must be less than or equal to 'writeIndex' and 'boundary'.
fromPosition
,
int
the position to move bytes to. Must be greater than or equal to 'readIndex'.
toPosition
)

Moves all bytes starting at 'fromPosition' to 'toPosition', overwriting the bytes in-between.

pack-priv void
copyBytes(int
the start position from which to read.
position
,
byte[]
the buffer to copy into.
destination
,
int
the offset of the buffer to copy into.
destinationOffset
,
int
the number of bytes to copy.
length
)

Copies bytes from the underlying buffer.

public void
copyTo(final OutputStream
stream to which the bytes will be copied.
outputStream
)

Copies all of the available bytes in the buffer without changing the number of bytes available to subsequent reads.

private void
ensureSpaceInBuffer(int
the minimum amount of free space that needs to be available for writing.
minimumNumberOfBytesRequired
)

Ensures that there is at least 'minimumNumberOfBytesRequired' bytes of free space in the buffer, growing the buffer if necessary.

pack-priv void
extendBoundary(int
the number of bytes by which the boundary should be extended.
numberOfBytes
)

Extends the boundary by the given number of bytes.

private int

Returns:

the number of bytes that can be written at the end of the buffer.
freeSpaceAtEndOfBuffer
()

pack-priv int

Returns:

the index of the boundary, which is used to mark the last buffered byte that is available for reading. The boundary must always fall within [readIndex, writeIndex].
getBoundary
()

pack-priv ByteBuffer

Returns:

a ByteBuffer.
getByteBuffer
(int
the start position of the ByteBuffer.
position
,
int
the limit of the ByteBuffer.
limit
)

Returns a ByteBuffer view of the underlying buffer.

pack-priv int

Returns:

the initial capacity of the buffer.
getInitialBufferSize
()

pack-priv int

Returns:

the current read index.
getReadIndex
()

pack-priv int

Returns:

the current write index.
getWriteIndex
()

private void
moveBytesToStartOfBuffer(byte[]
the destination buffer, which may be 'buffer' itself or a new buffer.
destinationBuffer
)

Moves all buffered (but not yet read) bytes from 'buffer' to the destination buffer.

pack-priv int

Returns:

the byte value.
peek
(int
the index of the byte to peek.
index
)

Peeks the byte at the given index without modifying any internal indexes.

public int
read(final byte[]
the buffer into which the data is read.
b
,
final int
the start offset in array b at which the data is written.
off
,
final int
the maximum number of bytes to read.
len
)

Overrides java.io.InputStream.read.

Reads up to len bytes of data from the input stream into an array of bytes.

public int
read()

Implements abstract java.io.InputStream.read.

Reads the next byte of data from the input stream.

public void
receive(final int
the byte to buffer.
b
)

Buffers a single additional byte, growing the buffer if it is already full.

public void
receive(final byte[]
the bytes to buffer.
b
,
final int
the offset into `b` that points to the first byte to buffer.
off
,
final int
the number of bytes to buffer.
len
)

Buffers `len` additional bytes, growing the buffer if it is already full or if it would become full by writing `len` bytes.

public void
receive(final byte[]
the bytes to buffer.
b
)

Buffers `b.length` additional bytes.

public int

Returns:

the number of bytes actually written, which will only be less than `len` if InputStream#read(byte[], int, int) returns less than `len`.
receive
(final InputStream
the source of the bytes.
input
,
final int
the number of bytes to attempt to write.
len
)

Buffers up to `len` additional bytes, growing the buffer if it is already full or if it would become full by writing `len` bytes.

pack-priv void
registerNotificationConsumer(ResizingPipedInputStream.NotificationConsumer
the NotificationConsumer to register.
consumer
)

Registers the given NotificationConsumer.

pack-priv void
rewind(final int
the read index value to be set.
previousReadIndex
,
final int
the available value to be set.
previousAvailable
)

Rewinds the buffer to the given read index and sets 'available' to the given value.

pack-priv void
seekTo(int
the index to which to seek. Must not be negative.
index
)

Seeks the read index to the given position.

pack-priv int

Returns:

the number of bytes actually buffered, which will be greater than or equal to 'available' if a boundary has been set, or equal to `available` if no boundary has been set.
size
()

public long

Returns:

the number of bytes actually skipped.
skip
(final long
the number of bytes to skip.
n
)

Overrides java.io.InputStream.skip.

Skips up to `n` buffered bytes.

pack-priv void
truncate(final int
the write index value to be set.
previousWriteIndex
,
final int
the available value to be set.
previousAvailable
)

Truncates the buffer to the given write index and sets both 'available' and 'size' to the to the given value.

Inherited from java.io.InputStream:
closemarkmarkSupportednullInputStreamreadreadAllBytesreadNBytesreadNBytesresetskipNBytestransferTo

Field Detail

availableback to summary
private int available
See Also
available()
boundaryback to summary
private int boundary
See Also
getBoundary()
bufferback to summary
private byte[] buffer

The raw buffer.

byteBufferback to summary
private ByteBuffer byteBuffer

View to the raw buffer.

capacityback to summary
private int capacity
See Also
capacity()
initialBufferSizeback to summary
private final int initialBufferSize

The initial size of the buffer and the number of bytes by which the size of the buffer will increase each time it grows, unless it must grow by a smaller amount to fit within 'maximumBufferSize'.

maximumBufferSizeback to summary
private final int maximumBufferSize

The maximum size of the buffer. If the user attempts to buffer more bytes than this, an exception will be raised.

NO_OP_NOTIFICATION_CONSUMERback to summary
private static final ResizingPipedInputStream.NotificationConsumer NO_OP_NOTIFICATION_CONSUMER

A NotificationConsumer that does nothing.

notificationConsumerback to summary
private ResizingPipedInputStream.NotificationConsumer notificationConsumer

The NotificationConsumer currently registered.

readIndexback to summary
private int readIndex

The index of the next byte in the buffer that is available to be read. Always less than or equal to `writeIndex`.

SINGLE_BYTE_MASKback to summary
private static final int SINGLE_BYTE_MASK

Mask to isolate a single byte.

sizeback to summary
private int size
See Also
size()
useBoundaryback to summary
private final boolean useBoundary

Whether to use a boundary to limit the number of available bytes. This can be used to buffer arbitrarily-sized chunks of bytes without making them available for consumption. When true, the boundary must be manually extended (see extendBoundary(int) to make these bytes available. When false, all buffered bytes will be available to read.

writeIndexback to summary
private int writeIndex

The index at which the next byte received will be written. Always greater than or equal to `readIndex`.

Constructor Detail

ResizingPipedInputStreamback to summary
public ResizingPipedInputStream(final int initialBufferSize)

Constructor.

Parameters
initialBufferSize:int

the initial size of the buffer. When full, the buffer will grow by this many bytes. The buffer always stores bytes contiguously, so growth requires allocation of a new buffer capable of holding the new capacity and copying of the existing bytes into the new buffer. As such, a size should be chosen carefully such that growth is expected to occur rarely, if ever.

ResizingPipedInputStreamback to summary
pack-priv ResizingPipedInputStream(final int initialBufferSize, final int maximumBufferSize, final boolean useBoundary)

Constructor.

Parameters
initialBufferSize:int

the initial size of the buffer. When full, the buffer will grow by this many bytes. The buffer always stores bytes contiguously, so growth requires allocation of a new buffer capable of holding the new capacity and copying of the existing bytes into the new buffer. As such, a size should be chosen carefully such that growth is expected to occur rarely, if ever.

maximumBufferSize:int

the maximum size of the buffer. If a call to `receive` attempts to transfer an amount of bytes that would cause the buffer to exceed this size, a BufferOverflowException will be thrown. Must be greater than or equal to the initial buffer size.

useBoundary:boolean

whether to use a boundary to limit the number of available bytes. This can be used to buffer arbitrarily-sized chunks of bytes without making them available for consumption. When true, the boundary must be manually extended (see extendBoundary(int) to make these bytes available. When false, all buffered bytes will be available to read.

Method Detail

availableback to summary
public int available()

Overrides java.io.InputStream.available.

Doc from java.io.InputStream.available.

Returns an estimate of the number of bytes that can be read (or skipped over) from this input stream without blocking, which may be 0, or 0 when end of stream is detected. The read might be on the same thread or another thread. A single read or skip of this many bytes will not block, but may read or skip fewer bytes.

Note that while some implementations of InputStream will return the total number of bytes in the stream, many will not. It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream.

A subclass's implementation of this method may choose to throw an IOException if this input stream has been closed by invoking the close() method.

Note

This method adheres to the documented behavior of InputStream#available() except that it always returns the exact number of bytes that are available in the buffer.

Returns:int

the exact number of bytes available in the buffer.

Annotations
@Override
availableBeyondBoundaryback to summary
pack-priv int availableBeyondBoundary()
Returns:int

the number of bytes buffered beyond the boundary. This is equivalent to subtracting available() from size().

capacityback to summary
public int capacity()
Returns:int

the capacity of the buffer, which is always less than or equal to 'maximumBufferSize'.

clearback to summary
pack-priv void clear()

Clears the buffer.

consolidateback to summary
pack-priv void consolidate(int fromPosition, int toPosition)

Moves all bytes starting at 'fromPosition' to 'toPosition', overwriting the bytes in-between. It is the caller's responsibility to ensure that the overwritten bytes are not needed.

Parameters
fromPosition:int

the position to move bytes from. Must be less than or equal to 'writeIndex' and 'boundary'.

toPosition:int

the position to move bytes to. Must be greater than or equal to 'readIndex'.

copyBytesback to summary
pack-priv void copyBytes(int position, byte[] destination, int destinationOffset, int length)

Copies bytes from the underlying buffer. It is the caller's responsibility to ensure the requested bytes are available.

Parameters
position:int

the start position from which to read.

destination:byte[]

the buffer to copy into.

destinationOffset:int

the offset of the buffer to copy into.

length:int

the number of bytes to copy.

copyToback to summary
public void copyTo(final OutputStream outputStream) throws IOException

Copies all of the available bytes in the buffer without changing the number of bytes available to subsequent reads.

Parameters
outputStream:OutputStream

stream to which the bytes will be copied.

Exceptions
IOException:
if thrown by OutputStream#write(byte[], int, int).
ensureSpaceInBufferback to summary
private void ensureSpaceInBuffer(int minimumNumberOfBytesRequired)

Ensures that there is at least 'minimumNumberOfBytesRequired' bytes of free space in the buffer, growing the buffer if necessary. May consolidate buffered bytes, performing an in-order copy and resetting indices such that the `readIndex` points to the same byte and the `writeIndex` is positioned after the last byte that is available to read.

Parameters
minimumNumberOfBytesRequired:int

the minimum amount of free space that needs to be available for writing.

extendBoundaryback to summary
pack-priv void extendBoundary(int numberOfBytes)

Extends the boundary by the given number of bytes. It is the caller's responsibility to ensure that the resulting boundary includes only bytes that have been buffered (i.e. that it does not exceed `writeIndex`).

Parameters
numberOfBytes:int

the number of bytes by which the boundary should be extended.

freeSpaceAtEndOfBufferback to summary
private int freeSpaceAtEndOfBuffer()
Returns:int

the number of bytes that can be written at the end of the buffer.

getBoundaryback to summary
pack-priv int getBoundary()
Returns:int

the index of the boundary, which is used to mark the last buffered byte that is available for reading. The boundary must always fall within [readIndex, writeIndex].

getByteBufferback to summary
pack-priv ByteBuffer getByteBuffer(int position, int limit)

Returns a ByteBuffer view of the underlying buffer.

Parameters
position:int

the start position of the ByteBuffer.

limit:int

the limit of the ByteBuffer.

Returns:ByteBuffer

a ByteBuffer.

getInitialBufferSizeback to summary
pack-priv int getInitialBufferSize()
Returns:int

the initial capacity of the buffer.

getReadIndexback to summary
pack-priv int getReadIndex()
Returns:int

the current read index.

getWriteIndexback to summary
pack-priv int getWriteIndex()
Returns:int

the current write index.

moveBytesToStartOfBufferback to summary
private void moveBytesToStartOfBuffer(byte[] destinationBuffer)

Moves all buffered (but not yet read) bytes from 'buffer' to the destination buffer. In total, size() bytes will be moved.

Parameters
destinationBuffer:byte[]

the destination buffer, which may be 'buffer' itself or a new buffer.

peekback to summary
pack-priv int peek(int index)

Peeks the byte at the given index without modifying any internal indexes. It is the caller's responsibility to ensure that the given index points to an available byte.

Parameters
index:int

the index of the byte to peek.

Returns:int

the byte value.

readback to summary
public int read(final byte[] b, final int off, final int len)

Overrides java.io.InputStream.read.

Doc from java.io.InputStream.read.

Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read. The number of bytes actually read is returned as an integer.

This method blocks until input data is available, end of file is detected, or an exception is thrown.

If len is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at end of file, the value -1 is returned; otherwise, at least one byte is read and stored into b.

The first byte read is stored into element b[off], the next one into b[off+1], and so on. The number of bytes read is, at most, equal to len. Let k be the number of bytes actually read; these bytes will be stored in elements b[off] through b[off+k-1], leaving elements b[off+k] through b[off+len-1] unaffected.

In every case, elements b[0] through b[off-1] and elements b[off+len] through b[b.length-1] are unaffected.

Note

This method adheres to the documented behavior of InputStream#read(byte[], int, int) except that it never blocks. If a read is attempted before the first write, this method will return -1.

Parameters
b:byte[]

the buffer into which the data is read.

off:int

the start offset in array b at which the data is written.

len:int

the maximum number of bytes to read.

Returns:int

the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached.

Annotations
@Override
readback to summary
public int read()

Implements abstract java.io.InputStream.read.

Doc from java.io.InputStream.read.

Reads the next byte of data from the input stream. The value byte is returned as an int in the range 0 to 255. If no byte is available because the end of the stream has been reached, the value -1 is returned. This method blocks until input data is available, the end of the stream is detected, or an exception is thrown.

Note

This method adheres to the documented behavior of InputStream#read(byte[], int, int) except that it never blocks. If a read is attempted before the first write, this method will return -1.

Returns:int

the next byte of data, or -1 if the end of the stream is reached.

Annotations
@Override
receiveback to summary
public void receive(final int b)

Buffers a single additional byte, growing the buffer if it is already full.

Parameters
b:int

the byte to buffer.

receiveback to summary
public void receive(final byte[] b, final int off, final int len)

Buffers `len` additional bytes, growing the buffer if it is already full or if it would become full by writing `len` bytes.

Parameters
b:byte[]

the bytes to buffer.

off:int

the offset into `b` that points to the first byte to buffer.

len:int

the number of bytes to buffer.

receiveback to summary
public void receive(final byte[] b)

Buffers `b.length` additional bytes.

Parameters
b:byte[]

the bytes to buffer.

See Also
receive(byte[], int, int)
receiveback to summary
public int receive(final InputStream input, final int len) throws IOException

Buffers up to `len` additional bytes, growing the buffer if it is already full or if it would become full by writing `len` bytes. This method will block if and only if the given `InputStream`'s InputStream#read(byte[], int, int) blocks when trying to read `len` bytes. If this is not desired, the caller should ensure that the given `InputStream` has at least `len` bytes available before calling this method or provide an InputStream implementation that does not block.

Parameters
input:InputStream

the source of the bytes.

len:int

the number of bytes to attempt to write.

Returns:int

the number of bytes actually written, which will only be less than `len` if InputStream#read(byte[], int, int) returns less than `len`.

Exceptions
IOException:
if thrown by the given `InputStream` during read, except for EOFException. If an EOFException is thrown by the `InputStream`, it will be caught and this method will return the number of bytes that were received before the exception was thrown.
registerNotificationConsumerback to summary
pack-priv void registerNotificationConsumer(ResizingPipedInputStream.NotificationConsumer consumer)

Registers the given NotificationConsumer.

Parameters
consumer:ResizingPipedInputStream.NotificationConsumer

the NotificationConsumer to register.

rewindback to summary
pack-priv void rewind(final int previousReadIndex, final int previousAvailable)

Rewinds the buffer to the given read index and sets 'available' to the given value. Subsequent behavior is undefined unless the values resulted from calling getReadIndex() and available() in immediate sequence, without any calls to 'receive' since.

Parameters
previousReadIndex:int

the read index value to be set.

previousAvailable:int

the available value to be set.

seekToback to summary
pack-priv void seekTo(int index)

Seeks the read index to the given position.

Parameters
index:int

the index to which to seek. Must not be negative.

sizeback to summary
pack-priv int size()
Returns:int

the number of bytes actually buffered, which will be greater than or equal to 'available' if a boundary has been set, or equal to `available` if no boundary has been set.

skipback to summary
public long skip(final long n)

Overrides java.io.InputStream.skip.

Skips up to `n` buffered bytes. Less than `n` bytes will be skipped if less than `n` bytes are available in the buffer.

Parameters
n:long

the number of bytes to skip.

Returns:long

the number of bytes actually skipped.

Annotations
@Override
truncateback to summary
pack-priv void truncate(final int previousWriteIndex, final int previousAvailable)

Truncates the buffer to the given write index and sets both 'available' and 'size' to the to the given value. Subsequent behavior is undefined unless the values resulted from calling getWriteIndex() and available() in immediate sequence, without any calls to 'read' or 'skip' since. It is the caller's responsibility to ensure that calling this method will not result in loss of important data beyond the boundary.

Parameters
previousWriteIndex:int

the write index value to be set.

previousAvailable:int

the available value to be set.

com.amazon.ion.impl back to summary

pack-priv Interface ResizingPipedInputStream.NotificationConsumer


Handler of notifications provided by the ResizingPipedInputStream.

Method Summary

Modifier and TypeMethod and Description
public void
bytesConsolidatedToStartOfBuffer(int
the amount of the left shift (also: the pre-shift read index of the first shifted byte).
leftShiftAmount
)

Bytes have been shifted to the start of the buffer in order to make room for additional bytes to be buffered.

Method Detail

bytesConsolidatedToStartOfBufferback to summary
public void bytesConsolidatedToStartOfBuffer(int leftShiftAmount)

Bytes have been shifted to the start of the buffer in order to make room for additional bytes to be buffered.

Parameters
leftShiftAmount:int

the amount of the left shift (also: the pre-shift read index of the first shifted byte).