Top Description Fields Constructors Methods
org.apache.derby.impl.store.raw.log

public Class LogAccessFile

extends Object
Class Inheritance
Imports
org.apache.derby.shared.common.reference.SQLState, org.apache.derby.shared.common.sanity.SanityManager, org.apache.derby.shared.common.error.StandardException, org.apache.derby.io.StorageRandomAccessFile, java.io.IOException, .OutputStream, .SyncFailedException, java.util.LinkedList, org.apache.derby.iapi.services.io.FormatIdOutputStream, .ArrayOutputStream, org.apache.derby.iapi.store.replication.master.MasterFactory, org.apache.derby.iapi.store.raw.RawStoreFactory, org.apache.derby.iapi.util.InterruptStatus
Wraps a RandomAccessFile file to provide buffering on log writes. Only supports the write calls required for the log! MT - unsafe. Caller of this class must provide synchronization. The one exception is with the log file access, LogAccessFile will touch the log only inside synchronized block protected by the semaphore, which is defined by the creator of this object. Write to the log buffers are allowed when there are free buffers even when dirty buffers are being written(flushed) to the disk by a different thread. Only one flush writes to log file at a time, other wait for it to finish. Except for flushLogAccessFile , SyncAccessLogFile other function callers must provide syncronization that will allow only one of them to write to the buffers. Log Buffers are used in circular fashion, each buffer moves through following stages: freeBuffers --> dirtyBuffers --> freeBuffers. Movement of buffers from one stage to another stage is synchronized using the object(this) of this class. A Checksum log record that has the checksum value for the data that is being written to the disk is generated and written before the actual data. Except for the large log records that does not fit into a single buffer, checksum is calcualted for a group of log records that are in the buffer when buffers is switched. Checksum log record is written into the reserved space in the beginning buffer. In case of a large log record that does not fit into a buffer, the checksum is written to the byte[] allocated for the big log record. Checksum log records helps in identifying the incomplete log disk writes during recovery. This is done by recalculating the checksum value for the data on the disk and comparing it to the the value stored in the checksum log record.

Field Summary

Modifier and TypeField and Description
private long
checksumInstant

private int
checksumLength

private ChecksumOperation
checksumLogOperation

private LogRecord
checksumLogRecord

private int
checksumLogRecordSize

private LogAccessFileBuffer
currentBuffer

private boolean
databaseEncrypted

private LinkedList<LogAccessFileBuffer>
dirtyBuffers

private boolean
flushInProgress

private LinkedList<LogAccessFileBuffer>
freeBuffers

pack-priv boolean
inReplicationMasterMode

pack-priv boolean
inReplicationSlaveMode

private final StorageRandomAccessFile
log

private static final int
LOG_NUMBER_LOG_BUFFERS

private static final int
LOG_RECORD_FIXED_OVERHEAD_SIZE

The fixed size of a log record is 16 bytes: int length : 4 bytes long instant : 8 bytes int trailing length : 4 bytes

private static final int
LOG_RECORD_HEADER_SIZE

private static final int
LOG_RECORD_TRAILER_SIZE

private LogToFile
logFactory

private final Object
logFileSemaphore

private FormatIdOutputStream
logicalOut

private ArrayOutputStream
logOutputBuffer

pack-priv MasterFactory
masterFac

pack-priv static int
mon_numBytesToLog

pack-priv static int
mon_numWritesToLog

private boolean
writeChecksum

Constructor Summary

AccessConstructor and Description
public
LogAccessFile(LogToFile logFactory, StorageRandomAccessFile log, int bufferSize)

Method Summary

Modifier and TypeMethod and Description
private int
appendLogRecordToBuffer(byte[]
The byte[] the log record is appended to
buff, int
The position in buff where the method will start to append to
pos, int
(data + optional_data) length bytes to write
length, long
the log address of this log record.
instant, byte[]
"from" array to copy "data" portion of rec
data, int
offset in "data" to start copying from.
data_offset, byte[]
"from" array to copy "optional data" from
optional_data, int
offset in "optional_data" to start copy from
optional_data_offset, int
length of optional data to copy.
optional_data_length)

Append a log record to a byte[].

public void
close()

public void
corrupt()

The database is being marked corrupted, get rid of file pointer without writing out anything more.

protected void
flushDirtyBuffers()

Write data from all dirty buffers into the log file.

public void
flushLogAccessFile()

public int
getChecksumLogRecordSize()

Return the length of a checksum record

protected long

Returns:

the space that is needed to write a checksum log record.
reserveSpaceForChecksum(int
the length of the log record to be written
length, long
current log file number
logFileNumber, long
current position in the log file.
currentPosition)

reserve the space for the checksum log record in the log file.

protected void
setReplicationMasterRole(MasterFactory
The MasterFactory service responsible for controlling the master side replication behaviour.
masterFac)

Make this LogAccessFile pass chunks of log records (byte[]) to the MasterFactory when the chunks are written to disk.

protected void
setReplicationSlaveRole()

Method to put this LogAccessFile object in replication slave mode, effectively disabling checksum writes.

protected void
stopReplicationMasterRole()

Stop this LogAccessFile from passing chunks of log records to the MasterFactory.

public void
switchLogBuffer()

Appends the current Buffer to the dirty Buffer list and assigns a free buffer to be the currrent active buffer .

public void
syncLogAccessFile()

Guarantee all writes up to the last call to flushLogAccessFile on disk.

public void
write(int b)

public void
write(byte[] b, int off, int len)

private void
writeChecksumLogRecord(byte[]
The byte[] the checksum is written to. The checksum is always written at the beginning of buffer.
buffer)

Generate the checkum log record and write it into the log buffer.

protected void
writeEndMarker(int marker)

private final int
writeInt(int i, byte[] b, int p)

public void
writeInt(int i)

public void
writeLogRecord(int
(data + optional_data) length bytes to write
length, long
the log address of this log record.
instant, byte[]
"from" array to copy "data" portion of rec
data, int
offset in "data" to start copying from.
data_offset, byte[]
"from" array to copy "optional data" from
optional_data, int
offset in "optional_data" to start copy from
optional_data_offset, int
length of optional data to copy.
optional_data_length)

Write a single log record to the stream.

private final int
writeLong(long l, byte[] b, int p)

public void
writeLong(long l)

private void
writeToLog(byte[] b, int off, int len, long highestInstant)

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

checksumInstantback to summary
private long checksumInstant
checksumLengthback to summary
private int checksumLength
checksumLogOperationback to summary
private ChecksumOperation checksumLogOperation
checksumLogRecordback to summary
private LogRecord checksumLogRecord
checksumLogRecordSizeback to summary
private int checksumLogRecordSize
currentBufferback to summary
private LogAccessFileBuffer currentBuffer
databaseEncryptedback to summary
private boolean databaseEncrypted
dirtyBuffersback to summary
private LinkedList<LogAccessFileBuffer> dirtyBuffers
flushInProgressback to summary
private boolean flushInProgress
freeBuffersback to summary
private LinkedList<LogAccessFileBuffer> freeBuffers
inReplicationMasterModeback to summary
pack-priv boolean inReplicationMasterMode
inReplicationSlaveModeback to summary
pack-priv boolean inReplicationSlaveMode
logback to summary
private final StorageRandomAccessFile log
LOG_NUMBER_LOG_BUFFERSback to summary
private static final int LOG_NUMBER_LOG_BUFFERS
LOG_RECORD_FIXED_OVERHEAD_SIZEback to summary
private static final int LOG_RECORD_FIXED_OVERHEAD_SIZE

The fixed size of a log record is 16 bytes: int length : 4 bytes long instant : 8 bytes int trailing length : 4 bytes

LOG_RECORD_HEADER_SIZEback to summary
private static final int LOG_RECORD_HEADER_SIZE
LOG_RECORD_TRAILER_SIZEback to summary
private static final int LOG_RECORD_TRAILER_SIZE
logFactoryback to summary
private LogToFile logFactory
logFileSemaphoreback to summary
private final Object logFileSemaphore
logicalOutback to summary
private FormatIdOutputStream logicalOut
logOutputBufferback to summary
private ArrayOutputStream logOutputBuffer
masterFacback to summary
pack-priv MasterFactory masterFac
mon_numBytesToLogback to summary
pack-priv static int mon_numBytesToLog
mon_numWritesToLogback to summary
pack-priv static int mon_numWritesToLog
writeChecksumback to summary
private boolean writeChecksum

Constructor Detail

LogAccessFileback to summary
public LogAccessFile(LogToFile logFactory, StorageRandomAccessFile log, int bufferSize)

Method Detail

appendLogRecordToBufferback to summary
private int appendLogRecordToBuffer(byte[] buff, int pos, int length, long instant, byte[] data, int data_offset, byte[] optional_data, int optional_data_offset, int optional_data_length)

Append a log record to a byte[]. Typically, the byte[] will be currentBuffer, but if a log record that is too big to fit in a buffer is added, buff will be a newly allocated byte[].

Parameters
buff:byte[]

The byte[] the log record is appended to

pos:int

The position in buff where the method will start to append to

length:int

(data + optional_data) length bytes to write

instant:long

the log address of this log record.

data:byte[]

"from" array to copy "data" portion of rec

data_offset:int

offset in "data" to start copying from.

optional_data:byte[]

"from" array to copy "optional data" from

optional_data_offset:int

offset in "optional_data" to start copy from

optional_data_length:int

length of optional data to copy.

See Also
LogAccessFile#writeLogRecord
closeback to summary
public void close() throws IOException, StandardException
corruptback to summary
public void corrupt() throws IOException

The database is being marked corrupted, get rid of file pointer without writing out anything more.

flushDirtyBuffersback to summary
protected void flushDirtyBuffers() throws IOException

Write data from all dirty buffers into the log file.

A call for clients of LogAccessFile to insure that all privately buffered data has been writen to the file - so that reads on the file using one of the various scan classes will see all the data which has been writen to this point.

Note that this routine only "writes" the data to the file, this does not mean that the data has been synced to disk unless file was opened in WRITE SYNC mode(rws/rwd). The only way to insure that is by calling is to call syncLogAccessFile() after this call in Non-WRITE sync mode(rw)

MT-Safe : parallel thereads can call this function, only one threads does the flush and the other threads waits for the one that is doing the flush to finish. Currently there are two possible threads that can call this function in parallel 1) A Thread that is doing the commit 2) A Thread that is writing to the log and log buffers are full or a log records does not fit in a buffer. (Log Buffers full(switchLogBuffer() or a log record size that is greater than logbuffer size has to be writtern through writeToLog call directlty) Note: writeToLog() is not synchronized on the semaphore that is used to do buffer management to allow writes to the free buffers when flush is in progress.

flushLogAccessFileback to summary
public void flushLogAccessFile() throws IOException, StandardException
getChecksumLogRecordSizeback to summary
public int getChecksumLogRecordSize()

Return the length of a checksum record

reserveSpaceForChecksumback to summary
protected long reserveSpaceForChecksum(int length, long logFileNumber, long currentPosition) throws StandardException, IOException

reserve the space for the checksum log record in the log file.

Parameters
length:int

the length of the log record to be written

logFileNumber:long

current log file number

currentPosition:long

current position in the log file.

Returns:long

the space that is needed to write a checksum log record.

setReplicationMasterRoleback to summary
protected void setReplicationMasterRole(MasterFactory masterFac)

Make this LogAccessFile pass chunks of log records (byte[]) to the MasterFactory when the chunks are written to disk.

Parameters
masterFac:MasterFactory

The MasterFactory service responsible for controlling the master side replication behaviour.

setReplicationSlaveRoleback to summary
protected void setReplicationSlaveRole()

Method to put this LogAccessFile object in replication slave mode, effectively disabling checksum writes. Because checksums are received from the replication master, the slave can not be allowed to add it's own checksums - that would invalidate the checksums and would stop the database from recovering. Replication slave mode must therefore be set before LogAccessFile decides whether to write it's own checksums, and this method is therefore indirectly called from the constructor of this class by calling LogFactory.checkForReplication If replication slave mode for the database is stopped after this object has been created, checksums cannot be reenabled without creating a new instance of this class. That is conveniently handled as LogToFile.recover completes (which automatically happens once replication slave mode is no longer active)

See Also
LogToFile#checkForReplication
stopReplicationMasterRoleback to summary
protected void stopReplicationMasterRole()

Stop this LogAccessFile from passing chunks of log records to the MasterFactory.

switchLogBufferback to summary
public void switchLogBuffer() throws IOException, StandardException

Appends the current Buffer to the dirty Buffer list and assigns a free buffer to be the currrent active buffer . Flushing of the buffer to disk is delayed if there is a free buffer available. dirty buffers will be flushed to the disk when flushDirtyBuffers() is invoked by a commit call or when no more free buffers are available.

syncLogAccessFileback to summary
public void syncLogAccessFile() throws IOException, StandardException

Guarantee all writes up to the last call to flushLogAccessFile on disk.

A call for clients of LogAccessFile to insure that all data written up to the last call to flushLogAccessFile() are written to disk. This call will not return until those writes have hit disk.

Note that this routine may block waiting for I/O to complete so callers should limit the number of resource held locked while this operation is called. It is expected that the caller Note that this routine only "writes" the data to the file, this does not mean that the data has been synced to disk. The only way to insure that is to first call switchLogBuffer() and then follow by a call of sync().

writeback to summary
public void write(int b)
writeback to summary
public void write(byte[] b, int off, int len)
writeChecksumLogRecordback to summary
private void writeChecksumLogRecord(byte[] buffer) throws IOException, StandardException

Generate the checkum log record and write it into the log buffer. The checksum applies to all bytes from this checksum log record to the next one.

Parameters
buffer:byte[]

The byte[] the checksum is written to. The checksum is always written at the beginning of buffer.

writeEndMarkerback to summary
protected void writeEndMarker(int marker) throws IOException, StandardException
writeIntback to summary
private final int writeInt(int i, byte[] b, int p)
writeIntback to summary
public void writeInt(int i)
writeLogRecordback to summary
public void writeLogRecord(int length, long instant, byte[] data, int data_offset, byte[] optional_data, int optional_data_offset, int optional_data_length) throws StandardException, IOException

Write a single log record to the stream.

For performance pass all parameters rather into a specialized routine rather than maintaining the writeInt, writeLong, and write interfaces that this class provides as a standard OutputStream. It will make it harder to use other OutputStream implementations, but makes for less function calls and allows optimizations knowing when to switch buffers.

This routine handles all log records which are smaller than one log buffer. If a log record is bigger than a log buffer it calls writeUnbufferedLogRecord().

The log record written will always look the same as if the following code had been executed: writeInt(length) writeLong(instant) write(data, data_offset, (length - optional_data_length) ) if (optional_data_length != 0) write(optional_data, optional_data_offset, optional_data_length) writeInt(length)

Parameters
length:int

(data + optional_data) length bytes to write

instant:long

the log address of this log record.

data:byte[]

"from" array to copy "data" portion of rec

data_offset:int

offset in "data" to start copying from.

optional_data:byte[]

"from" array to copy "optional data" from

optional_data_offset:int

offset in "optional_data" to start copy from

optional_data_length:int

length of optional data to copy.

Exceptions
StandardException:
Standard exception policy.
writeLongback to summary
private final int writeLong(long l, byte[] b, int p)
writeLongback to summary
public void writeLong(long l)
writeToLogback to summary
private void writeToLog(byte[] b, int off, int len, long highestInstant) throws IOException