Top Description Fields Constructors Methods
org.apache.derby.impl.store.raw.data

pack-priv Class RAFContainer4

extends RAFContainer
Class Inheritance
Imports
org.apache.derby.shared.common.error.StandardException, org.apache.derby.shared.common.reference.SQLState, org.apache.derby.shared.common.sanity.SanityManager, org.apache.derby.iapi.store.raw.ContainerKey, org.apache.derby.iapi.util.InterruptStatus, .InterruptDetectedException, java.io.EOFException, .IOException, .RandomAccessFile, java.nio.ByteBuffer, java.nio.channels.FileChannel, .ClosedChannelException, .ClosedByInterruptException, .AsynchronousCloseException, org.apache.derby.io.StorageRandomAccessFile

RAFContainer4 overrides a few methods in FileContainer/RAFContainer in order to use FileChannel from Java 1.4's New IO framework to issue multiple IO operations to the same file concurrently instead of strictly serializing IO operations using a mutex on the container object. Since we compile with Java 1.4, the override "annotations" are inside the method javadoc headers.

Note that our requests for multiple concurrent IOs may be serialized further down in the IO stack - this is entirely up to the JVM and OS. However, at least in Linux on Sun's 1.4.2_09 JVM we see the desired behavior: The FileChannel.read/write(ByteBuffer buf, long position) calls map to pread/pwrite system calls, which enable efficient IO to the same file descriptor by multiple threads.

This whole class should be merged back into RAFContainer when Derby officially stops supporting Java 1.3.

Significant behavior changes from RAFContainer:

  1. Multiple concurrent IOs permitted.
  2. State changes to the container (create, open, close) can now happen while IO is in progress due to the lack of locking. Closing a container while IO is in progress will cause IOExceptions in the thread calling readPage or writePage. If this happens something is probably amiss anyway. The iosInProgress variable is used in an attempt to detect this should it happen while running a debug build.
See Also
java.nio.channels.FileChannel

Field Summary

Modifier and TypeField and Description
private final Object
private ContainerKey
private boolean
private final Object
private int
iosInProgress

For debugging - will be incremented when an IO is started, decremented when it is done.

private FileChannel
ourChannel

This channel will be retrieved from RAFContainer's fileData member when fileData is set.

private volatile boolean
private volatile int
Inherited from org.apache.derby.impl.store.raw.data.RAFContainer:
fileDataneedsSync

Constructor Summary

AccessConstructor and Description
public

Method Summary

Modifier and TypeMethod and Description
private void
awaitRestoreChannel(Exception
the exception we saw during IO
e
,
boolean
true if the thread doing IO in stealth mode
stealthMode
)

Use when seeing an exception during IO and when another thread is presumably doing the recovery.

pack-priv synchronized void
closeContainer()

Overrides org.apache.derby.impl.store.raw.data.RAFContainer.closeContainer.

override of RAFContainer#closeContainer

pack-priv synchronized void
createContainer(ContainerKey newIdentity)

Overrides org.apache.derby.impl.store.raw.data.RAFContainer.createContainer.

override of RAFContainer#createContainer

private static void
private FileChannel

Returns:

a FileChannel if file is an instance of RandomAccessFile, null otherwise
getChannel
(StorageRandomAccessFile
the file to get the channel for
file
)

Return the FileChannel for the specified StorageRandomAccessFile if it is a RandomAccessFile.

private FileChannel

Returns:

a FileChannel object, if supported, or null
getChannel
()

Return the file channel for the current value of the fileData field.

pack-priv byte[]

Returns:

a byte array containing the embryonic page
getEmbryonicPage
(StorageRandomAccessFile
the file to read from
file
,
long
where to start reading (normally FileContainer.FIRST_ALLOC_PAGE_OFFSET)
offset
)

Overrides org.apache.derby.impl.store.raw.data.FileContainer.getEmbryonicPage.

Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in a StorageRandomAccessFile.

private void
handleClosedChannel(ClosedChannelException
Should be an instance of ClosedChannelException.
e
,
boolean
If true, do retry at a higher level
stealthMode
,
int
Give up waiting for another thread to reopen the channel when retries reaches 0. Only applicable if stealthMode == false.
retries
)

This method handles what to do when, during a NIO operation we receive a ClosedChannelException.

pack-priv synchronized boolean
private void
readFull(ByteBuffer
buffer to read into
dstBuffer
,
FileChannel
channel to read from
srcChannel
,
long
file position from where to read
position
)

Attempts to fill buf completely from start until it's full.

protected void
readPage(long pageNumber, byte[] pageData)

Overrides org.apache.derby.impl.store.raw.data.RAFContainer.readPage.

Read a page into the supplied array.

private void
readPage(long
the page number to read data from, or -1 (called from getEmbryonicPage)
pageNumber
,
byte[]
the buffer to read data into
pageData
,
long
-1 normally (not used since offset is computed from pageNumber), but used if pageNumber == -1 (getEmbryonicPage)
offset
)

Read a page into the supplied array.

private void
readPage0(long pageNumber, byte[] pageData, long offset)

private boolean

Returns:

true if we did recovery, false if we saw someone else do it and abstained
recoverContainerAfterInterrupt
(String
caller site (debug info)
whence
,
boolean
don't update threadsInPageIO if true
stealthMode
)

Use this when the thread has received a ClosedByInterruptException (or, prior to JDK 1.7 it may also be AsynchronousCloseException - a bug) exception during IO and its interruped flag is also set.

private void
reopen()

When the existing channel (ourChannel) has been closed due to interrupt, we need to reopen the underlying RAF to get a fresh channel so we can resume IO.

pack-priv void
writeAtOffset(StorageRandomAccessFile
the file to write to
file
,
byte[]
the bytes to write
bytes
,
long
the offset to start writing at
offset
)

Overrides org.apache.derby.impl.store.raw.data.FileContainer.writeAtOffset.

Write a sequence of bytes at the given offset in a file.

private void
writeFull(ByteBuffer
buffer to write
srcBuffer
,
FileChannel
channel to write to
dstChannel
,
long
file position to start writing at
position
)

Attempts to write buf completely from start until end, at the given position in the destination fileChannel.

protected void
writePage(long pageNumber, byte[] pageData, boolean syncPage)

Overrides org.apache.derby.impl.store.raw.data.RAFContainer.writePage.

Write a page from the supplied array.

private void
writePage0(long pageNumber, byte[] pageData, boolean syncPage)

Inherited from org.apache.derby.impl.store.raw.data.RAFContainer:
backupContainercleanencryptOrDecryptContainerflushAllgetFileNamegetRandomAccessFileisDirtypreAllocateprivGetFileNameremoveContainerremoveFilereopenContainerruntruncatePagesupdatePageArray

Field Detail

channelCleanupMonitorback to summary
private final Object channelCleanupMonitor
currentIdentityback to summary
private ContainerKey currentIdentity
giveUpIOback to summary
private boolean giveUpIO
giveUpIOmback to summary
private final Object giveUpIOm
iosInProgressback to summary
private int iosInProgress

For debugging - will be incremented when an IO is started, decremented when it is done. Should be == 0 when container state is changed.

ourChannelback to summary
private FileChannel ourChannel

This channel will be retrieved from RAFContainer's fileData member when fileData is set. We wrap a couple of RAFContainer's methods to accomplish this.

restoreChannelInProgressback to summary
private volatile boolean restoreChannelInProgress
threadsInPageIOback to summary
private volatile int threadsInPageIO

Constructor Detail

RAFContainer4back to summary
public RAFContainer4(BaseDataFileFactory factory)

Method Detail

awaitRestoreChannelback to summary
private void awaitRestoreChannel(Exception e, boolean stealthMode) throws StandardException

Use when seeing an exception during IO and when another thread is presumably doing the recovery.

If stealthMode == false, wait for another thread to recover the container after an interrupt. If stealthMode == true, throw internal exception InterruptDetectedException to do retry from higher in the stack.

If stealthMode == false, maximum wait time for the container to become available again is determined by the product InterruptStatus.MAX_INTERRUPT_RETRIES * InterruptStatus.INTERRUPT_RETRY_SLEEP. There is a chance this thread will not see any recovery occuring (yet), in which case it waits for a bit and just returns, so the caller must retry IO until success.

If for some reason the recovering thread has given up on resurrecting the container, cf #giveUpIO, the method throws FILE_IO_INTERRUPTED.

Parameters
e:Exception

the exception we saw during IO

stealthMode:boolean

true if the thread doing IO in stealth mode

Exceptions
StandardException:
InterruptDetectedException and normal error policy
closeContainerback to summary
pack-priv synchronized void closeContainer()

Overrides org.apache.derby.impl.store.raw.data.RAFContainer.closeContainer.

override of RAFContainer#closeContainer

createContainerback to summary
pack-priv synchronized void createContainer(ContainerKey newIdentity) throws StandardException

Overrides org.apache.derby.impl.store.raw.data.RAFContainer.createContainer.

override of RAFContainer#createContainer

Exceptions
StandardException:

Doc from org.apache.derby.impl.store.raw.data.FileContainer.createContainer.

Derby Standard error policy

debugTraceback to summary
private static void debugTrace(String msg)
getChannelback to summary
private FileChannel getChannel(StorageRandomAccessFile file)

Return the FileChannel for the specified StorageRandomAccessFile if it is a RandomAccessFile. Otherwise, return null.

Parameters
file:StorageRandomAccessFile

the file to get the channel for

Returns:FileChannel

a FileChannel if file is an instance of RandomAccessFile, null otherwise

getChannelback to summary
private FileChannel getChannel()

Return the file channel for the current value of the fileData field. If fileData doesn't support file channels, return null.

Callers of this method must synchronize on the container object since two shared fields (fileData and ourChannel) are accessed.

Returns:FileChannel

a FileChannel object, if supported, or null

getEmbryonicPageback to summary
pack-priv byte[] getEmbryonicPage(StorageRandomAccessFile file, long offset) throws IOException, StandardException

Overrides org.apache.derby.impl.store.raw.data.FileContainer.getEmbryonicPage.

Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in a StorageRandomAccessFile.

override of FileContainer#getEmbryonicPage

Parameters
file:StorageRandomAccessFile

the file to read from

offset:long

where to start reading (normally FileContainer.FIRST_ALLOC_PAGE_OFFSET)

Returns:byte[]

a byte array containing the embryonic page

Exceptions
IOException:
if an I/O error occurs while reading
StandardException:
if thread is interrupted.
handleClosedChannelback to summary
private void handleClosedChannel(ClosedChannelException e, boolean stealthMode, int retries) throws StandardException

This method handles what to do when, during a NIO operation we receive a ClosedChannelException. Note the specialization hierarchy:

ClosedChannelException -> AsynchronousCloseException -> ClosedByInterruptException

If e is a ClosedByInterruptException, we normally start container recovery, i.e. we need to reopen the random access file so we get get a new interruptible channel and continue IO.

If e is a AsynchronousCloseException or a plain ClosedChannelException, the behavior depends of stealthMode:

If stealthMode == false, the method will wait for another thread tp finish recovering the IO channel before returning.

If stealthMode == true, the method throws InterruptDetectedException, allowing retry at a higher level in the code. The reason for this is that we sometimes need to release monitors on objects needed by the recovery thread.

Parameters
e:ClosedChannelException

Should be an instance of ClosedChannelException.

stealthMode:boolean

If true, do retry at a higher level

retries:int

Give up waiting for another thread to reopen the channel when retries reaches 0. Only applicable if stealthMode == false.

Exceptions
StandardException:
standard error policy, incl. when we give up waiting for another thread to reopen channel
InterruptDetectedException:
if retry at higher level is required stealthMode == true.
openContainerback to summary
pack-priv synchronized boolean openContainer(ContainerKey newIdentity) throws StandardException

Overrides org.apache.derby.impl.store.raw.data.RAFContainer.openContainer.

Doc from org.apache.derby.impl.store.raw.data.FileContainer.openContainer.

Open a container.

Longer descrption of routine.

Open a container. Open the file that maps to this container, if the file does not exist then we assume the container was never created. If the file exists but we have trouble opening it then we throw some exception.
MT - single thread required - Enforced by cache manager.

Exceptions
StandardException:
Standard exception policy.
readFullback to summary
private void readFull(ByteBuffer dstBuffer, FileChannel srcChannel, long position) throws IOException, StandardException

Attempts to fill buf completely from start until it's full.

FileChannel has no readFull() method, so we roll our own.

Parameters
dstBuffer:ByteBuffer

buffer to read into

srcChannel:FileChannel

channel to read from

position:long

file position from where to read

Exceptions
IOException:
if an I/O error occurs while reading
StandardException:
If thread is interrupted.
readPageback to summary
protected void readPage(long pageNumber, byte[] pageData) throws IOException, StandardException

Overrides org.apache.derby.impl.store.raw.data.RAFContainer.readPage.

Read a page into the supplied array.

override of RAFContainer#readPage


MT - thread safe

Exceptions
IOException:
exception reading page
StandardException:
Standard Derby error policy
readPageback to summary
private void readPage(long pageNumber, byte[] pageData, long offset) throws IOException, StandardException

Read a page into the supplied array.

override of RAFContainer#readPage


MT - thread safe

Parameters
pageNumber:long

the page number to read data from, or -1 (called from getEmbryonicPage)

pageData:byte[]

the buffer to read data into

offset:long

-1 normally (not used since offset is computed from pageNumber), but used if pageNumber == -1 (getEmbryonicPage)

Exceptions
IOException:
exception reading page
StandardException:
Standard Derby error policy
readPage0back to summary
private void readPage0(long pageNumber, byte[] pageData, long offset) throws IOException, StandardException
recoverContainerAfterInterruptback to summary
private boolean recoverContainerAfterInterrupt(String whence, boolean stealthMode) throws StandardException

Use this when the thread has received a ClosedByInterruptException (or, prior to JDK 1.7 it may also be AsynchronousCloseException - a bug) exception during IO and its interruped flag is also set. This makes this thread a likely candicate to do container recovery, unless another thread started it already, cf. return value.

Parameters
whence:String

caller site (debug info)

stealthMode:boolean

don't update threadsInPageIO if true

Returns:boolean

true if we did recovery, false if we saw someone else do it and abstained

reopenback to summary
private void reopen() throws StandardException

When the existing channel (ourChannel) has been closed due to interrupt, we need to reopen the underlying RAF to get a fresh channel so we can resume IO.

writeAtOffsetback to summary
pack-priv void writeAtOffset(StorageRandomAccessFile file, byte[] bytes, long offset) throws IOException, StandardException

Overrides org.apache.derby.impl.store.raw.data.FileContainer.writeAtOffset.

Write a sequence of bytes at the given offset in a file. This method operates in stealth mode, see doc for handleClosedChannel. This presumes that IO retry happens at a higher level, i.e. the caller(s) must be prepared to handle InterruptDetectedException.

This method overrides FileContainer#writeAtOffset.

Parameters
file:StorageRandomAccessFile

the file to write to

bytes:byte[]

the bytes to write

offset:long

the offset to start writing at

Exceptions
IOException:
if an I/O error occurs while writing
StandardException:

Doc from org.apache.derby.impl.store.raw.data.FileContainer.writeAtOffset.

Derby Standard error policy

writeFullback to summary
private void writeFull(ByteBuffer srcBuffer, FileChannel dstChannel, long position) throws IOException

Attempts to write buf completely from start until end, at the given position in the destination fileChannel.

FileChannel has no writeFull() method, so we roll our own.

Parameters
srcBuffer:ByteBuffer

buffer to write

dstChannel:FileChannel

channel to write to

position:long

file position to start writing at

Exceptions
IOException:
if an I/O error occurs while writing
StandardException:
If thread is interrupted.
writePageback to summary
protected void writePage(long pageNumber, byte[] pageData, boolean syncPage) throws IOException, StandardException

Overrides org.apache.derby.impl.store.raw.data.RAFContainer.writePage.

Write a page from the supplied array.

override of RAFContainer#writePage


MT - thread safe

Exceptions
IOException:
IO error accessing page
StandardException:
Standard Derby error policy
writePage0back to summary
private void writePage0(long pageNumber, byte[] pageData, boolean syncPage) throws IOException, StandardException