Apache PDFBox 3.0.4 · org.apache.pdfbox.io.ScratchFile

Implements a memory page handling mechanism as base for creating (multiple) RandomAccess buffers each having its set of pages (implemented by ScratchFileBuffer). A buffer is created calling createBuffer().

Pages can be stored in main memory or in a temporary file. A mixed mode is supported storing a certain amount of pages in memory and only the additional ones in temporary file (defined by maximum main memory to be used).

Pages can be marked as 'free' in order to re-use them. For in-memory pages this will release the used memory while for pages in temporary file this simply marks the area as free to re-use.

If a temporary file was created (done with the first page to be stored in temporary file) it is deleted when ScratchFile#close() is called.

Using this class for RandomAccess buffers allows for a direct control on the maximum memory usage and allows processing large files for which we otherwise would get an OutOfMemoryError in case of using RandomAccessReadBuffer.

This base class for providing pages is thread safe (the buffer implementations are not).

Field Summary

Modifier and Type	Field and Description
private final List<ScratchFileBuffer>	buffers
private static final int	ENLARGE_PAGE_COUNT number of pages by which we enlarge the scratch file (reduce I/O-operations)
private File	file scratch file; only to be accessed under synchronization of `ioLock`
private final BitSet	freePages
private static final int	INIT_UNRESTRICTED_MAINMEM_PAGECOUNT in case of unrestricted main memory usage this is the initial number of pages `inMemoryPages` is setup for
private final int	inMemoryMaxPageCount
private volatile byte[][]	inMemoryPages holds pointers to in-memory page content; will be initialized once in case of restricted main memory, otherwise it is enlarged as needed and first initialized to a size of `INIT_UNRESTRICTED_MAINMEM_PAGECOUNT`
private final Object	ioLock
private volatile boolean	isClosed
private static final Log	LOG
private final boolean	maxMainMemoryIsRestricted
private final int	maxPageCount
private static final int	PAGE_SIZE
private volatile int	pageCount
private RandomAccessFile	raf random access to scratch file; only to be accessed under synchronization of `ioLock`
private final File	scratchFileDirectory
private final boolean	useScratchFile

Constructor Summary

Access	Constructor and Description
public	ScratchFile(File The directory in which to create the scratch file or `null` to created it in the default temporary directory. scratchFileDirectory) Initializes page handler.
public	ScratchFile(MemoryUsageSetting set how memory/temporary files are used for buffering streams etc. memUsageSetting) Initializes page handler.

Method Summary

Modifier and Type	Method and Description
pack-priv void	checkClosed() Checks if this page handler has already been closed.
public void	close() Implements java.io.Closeable.close. Closes and deletes the temporary file.
public RandomAccess	Returns: A new buffer. createBuffer() Implements org.apache.pdfbox.io.RandomAccessStreamCache.createBuffer. Creates a new buffer using this page handler.
private void	enlarge() This will provide new free pages by either enlarging the scratch file by a number of pages defined by `ENLARGE_PAGE_COUNT` - in case scratch file usage is allowed - or increase the `inMemoryPages` array in case main memory was not restricted.
public static ScratchFile	Returns: instance configured to only use main memory with no size restriction getMainMemoryOnlyInstance() Getter for an instance using only unrestricted main memory for buffering (same as `new ScratchFile(MemoryUsageSetting.setupMainMemoryOnly())`).
public static ScratchFile	Returns: instance configured to only use main memory with no size restriction or null if cannot create scratch file getMainMemoryOnlyInstance(long maximum number of main-memory to be used; `-1` for no restriction; `0` will also be interpreted here as no restriction maxMainMemoryBytes) Getter for an instance to only use main-memory with the defined maximum.
pack-priv int	Returns: index of new page getNewPage() Returns a new free page, either from free page pool or by enlarging scratch file (may be created).
pack-priv int	Returns: byte size of a page getPageSize() Returns byte size of a page.
private void	initPages()
pack-priv void	markPagesAsFree(int[] pages indexes of pages to release pageIndexes, int off, int number of page indexes contained in provided array count) Allows a buffer which is cleared/closed to release its pages to be re-used.
pack-priv byte[]	Returns: byte array of size `PAGE_SIZE` filled with page data read from file readPage(int index of page to read pageIdx) Reads the page with specified index.
pack-priv void	removeBuffer(ScratchFileBuffer buffer)
pack-priv void	writePage(int index of page to write pageIdx, byte[] page to write (length has to be 4096) page) Writes updated page.

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

Field Detail

buffers	back to summary
private final List<ScratchFileBuffer> buffers

ENLARGE_PAGE_COUNT	back to summary
private static final int ENLARGE_PAGE_COUNT number of pages by which we enlarge the scratch file (reduce I/O-operations)

file	back to summary
private File file scratch file; only to be accessed under synchronization of `ioLock`

freePages	back to summary
private final BitSet freePages

INIT_UNRESTRICTED_MAINMEM_PAGECOUNT	back to summary
private static final int INIT_UNRESTRICTED_MAINMEM_PAGECOUNT in case of unrestricted main memory usage this is the initial number of pages `inMemoryPages` is setup for

inMemoryMaxPageCount	back to summary
private final int inMemoryMaxPageCount

inMemoryPages	back to summary
private volatile byte[][] inMemoryPages holds pointers to in-memory page content; will be initialized once in case of restricted main memory, otherwise it is enlarged as needed and first initialized to a size of `INIT_UNRESTRICTED_MAINMEM_PAGECOUNT`

ioLock	back to summary
private final Object ioLock

isClosed	back to summary
private volatile boolean isClosed

LOG	back to summary
private static final Log LOG

maxMainMemoryIsRestricted	back to summary
private final boolean maxMainMemoryIsRestricted

maxPageCount	back to summary
private final int maxPageCount

PAGE_SIZE	back to summary
private static final int PAGE_SIZE

pageCount	back to summary
private volatile int pageCount

raf	back to summary
private RandomAccessFile raf random access to scratch file; only to be accessed under synchronization of `ioLock`

scratchFileDirectory	back to summary
private final File scratchFileDirectory

useScratchFile	back to summary
private final boolean useScratchFile

Constructor Detail

ScratchFile back to summary

ScratchFile	back to summary
public ScratchFile(File scratchFileDirectory) throws IOException Initializes page handler. If a `scratchFileDirectory` is supplied, then the scratch file will be created in that directory. All pages will be stored in the scratch file. Parameters scratchFileDirectory:File The directory in which to create the scratch file or `null` to created it in the default temporary directory. Exceptions IOException: If scratch file directory was given but don't exist.

public ScratchFile(File scratchFileDirectory) throws IOException

Initializes page handler. If a scratchFileDirectory is supplied, then the scratch file will be created in that directory.

All pages will be stored in the scratch file.

Parameters

scratchFileDirectory:File: The directory in which to create the scratch file or null to created it in the default temporary directory.

Exceptions

IOException:: If scratch file directory was given but don't exist.

ScratchFile back to summary

ScratchFile	back to summary
public ScratchFile(MemoryUsageSetting memUsageSetting) throws IOException Initializes page handler. If a `scratchFileDirectory` is supplied, then the scratch file will be created in that directory. Depending on the size of allowed memory usage a number of pages (memorySize/`PAGE_SIZE`) will be stored in-memory and only additional pages will be written to/read from scratch file. Parameters memUsageSetting:MemoryUsageSetting set how memory/temporary files are used for buffering streams etc. Exceptions IOException: If scratch file directory was given but don't exist.

public ScratchFile(MemoryUsageSetting memUsageSetting) throws IOException

Initializes page handler. If a scratchFileDirectory is supplied, then the scratch file will be created in that directory.

Depending on the size of allowed memory usage a number of pages (memorySize/PAGE_SIZE) will be stored in-memory and only additional pages will be written to/read from scratch file.

Parameters

memUsageSetting:MemoryUsageSetting: set how memory/temporary files are used for buffering streams etc.

Exceptions

IOException:: If scratch file directory was given but don't exist.

Method Detail

checkClosed back to summary

checkClosed	back to summary
pack-priv void checkClosed() throws IOException Checks if this page handler has already been closed. If so, an `IOException` is thrown. Exceptions IOException: If `close()` has already been called.

pack-priv void checkClosed() throws IOException

Checks if this page handler has already been closed. If so, an IOException is thrown.

Exceptions

IOException:: If close() has already been called.

close	back to summary
public void close() throws IOException Implements java.io.Closeable.close. Closes and deletes the temporary file. No further interaction with the scratch file or associated buffers can happen after this method is called. It also releases in-memory pages. Annotations @Override Exceptions IOException: If there was a problem closing or deleting the temporary file.

back to summary

public void close() throws IOException

Implements java.io.Closeable.close.

Closes and deletes the temporary file. No further interaction with the scratch file or associated buffers can happen after this method is called. It also releases in-memory pages.

Annotations

@Override

Exceptions

IOException:: If there was a problem closing or deleting the temporary file.

createBuffer	back to summary
public RandomAccess createBuffer() throws IOException Implements org.apache.pdfbox.io.RandomAccessStreamCache.createBuffer. Creates a new buffer using this page handler. Returns:RandomAccess A new buffer. Annotations @Override Exceptions IOException: If an error occurred.

createBuffer

back to summary

public RandomAccess createBuffer() throws IOException

Implements org.apache.pdfbox.io.RandomAccessStreamCache.createBuffer.

Creates a new buffer using this page handler.

Returns:RandomAccess

A new buffer.

Annotations

@Override

Exceptions

IOException:: If an error occurred.

enlarge back to summary

enlarge	back to summary
private void enlarge() throws IOException This will provide new free pages by either enlarging the scratch file by a number of pages defined by `ENLARGE_PAGE_COUNT` - in case scratch file usage is allowed - or increase the `inMemoryPages` array in case main memory was not restricted. If neither of both is allowed/the case than free pages count won't be changed. The same is true if no new pages could be added because we reached the maximum of `Integer#MAX_VALUE` pages. If scratch file usage is allowed and scratch file does not exist already it will be created. Only to be called under synchronization on `freePages`.

private void enlarge() throws IOException

This will provide new free pages by either enlarging the scratch file by a number of pages defined by ENLARGE_PAGE_COUNT - in case scratch file usage is allowed - or increase the inMemoryPages array in case main memory was not restricted. If neither of both is allowed/the case than free pages count won't be changed. The same is true if no new pages could be added because we reached the maximum of Integer#MAX_VALUE pages.

If scratch file usage is allowed and scratch file does not exist already it will be created.

Only to be called under synchronization on freePages.

getMainMemoryOnlyInstance back to summary

getMainMemoryOnlyInstance	back to summary
public static ScratchFile getMainMemoryOnlyInstance() Getter for an instance using only unrestricted main memory for buffering (same as `new ScratchFile(MemoryUsageSetting.setupMainMemoryOnly())`). Returns:ScratchFile instance configured to only use main memory with no size restriction

public static ScratchFile getMainMemoryOnlyInstance()

Getter for an instance using only unrestricted main memory for buffering (same as new ScratchFile(MemoryUsageSetting.setupMainMemoryOnly())).

Returns:ScratchFile: instance configured to only use main memory with no size restriction

getMainMemoryOnlyInstance back to summary

getMainMemoryOnlyInstance	back to summary
public static ScratchFile getMainMemoryOnlyInstance(long maxMainMemoryBytes) Getter for an instance to only use main-memory with the defined maximum. Parameters maxMainMemoryBytes:long maximum number of main-memory to be used; `-1` for no restriction; `0` will also be interpreted here as no restriction Returns:ScratchFile instance configured to only use main memory with no size restriction or null if cannot create scratch file

public static ScratchFile getMainMemoryOnlyInstance(long maxMainMemoryBytes)

Getter for an instance to only use main-memory with the defined maximum.

Parameters

maxMainMemoryBytes:long: maximum number of main-memory to be used; -1 for no restriction; 0 will also be interpreted here as no restriction

Returns:ScratchFile

instance configured to only use main memory with no size restriction or null if cannot create scratch file

getNewPage	back to summary
pack-priv int getNewPage() throws IOException Returns a new free page, either from free page pool or by enlarging scratch file (may be created). Returns:int index of new page

getNewPage

back to summary

pack-priv int getNewPage() throws IOException

Returns a new free page, either from free page pool or by enlarging scratch file (may be created).

Returns:int: index of new page

getPageSize	back to summary
pack-priv int getPageSize() Returns byte size of a page. Returns:int byte size of a page

getPageSize

back to summary

pack-priv int getPageSize()

Returns byte size of a page.

Returns:int: byte size of a page

initPages	back to summary
private void initPages()

markPagesAsFree	back to summary
pack-priv void markPagesAsFree(int[] pageIndexes, int off, int count) Allows a buffer which is cleared/closed to release its pages to be re-used. Parameters pageIndexes:int[] pages indexes of pages to release count:int number of page indexes contained in provided array

markPagesAsFree

back to summary

pack-priv void markPagesAsFree(int[] pageIndexes, int off, int count)

Allows a buffer which is cleared/closed to release its pages to be re-used.

Parameters

pageIndexes:int[]: pages indexes of pages to release
count:int: number of page indexes contained in provided array

readPage back to summary

readPage	back to summary
pack-priv byte[] readPage(int pageIdx) throws IOException Reads the page with specified index. Parameters pageIdx:int index of page to read Returns:byte[] byte array of size `PAGE_SIZE` filled with page data read from file

pack-priv byte[] readPage(int pageIdx) throws IOException

Reads the page with specified index.

Parameters

pageIdx:int: index of page to read

Returns:byte[]

byte array of size PAGE_SIZE filled with page data read from file

removeBuffer	back to summary
pack-priv void removeBuffer(ScratchFileBuffer buffer)

writePage back to summary

writePage	back to summary
pack-priv void writePage(int pageIdx, byte[] page) throws IOException Writes updated page. Page is either kept in-memory if pageIdx < `inMemoryMaxPageCount` or is written to scratch file. Provided page byte array must not be re-used for other pages since we store it as is in case of in-memory handling. Parameters pageIdx:int index of page to write page:byte[] page to write (length has to be 4096) Exceptions IOException: in case page index is out of range or page has wrong length or writing to file failed

pack-priv void writePage(int pageIdx, byte[] page) throws IOException

Writes updated page. Page is either kept in-memory if pageIdx < inMemoryMaxPageCount or is written to scratch file.

Provided page byte array must not be re-used for other pages since we store it as is in case of in-memory handling.

Parameters

pageIdx:int: index of page to write
page:byte[]: page to write (length has to be 4096)

Exceptions

IOException:: in case page index is out of range or page has wrong length or writing to file failed

public Class ScratchFile

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail