Top Description Fields Constructors Methods
org.apache.commons.compress.archivers.zip

public Class ParallelScatterZipCreator

extends Object
Class Inheritance
Static Imports
org.apache.commons.compress.archivers.zip.ZipArchiveEntryRequest.createZipArchiveEntryRequest
Creates a ZIP in parallel by using multiple threadlocal ScatterZipOutputStream instances.

Note that until 1.18, this class generally made no guarantees about the order of things written to the output file. Things that needed to come in a specific order (manifests, directories) had to be handled by the client of this class, usually by writing these things to the ZipArchiveOutputStream before calling writeTo on this class.

The client can supply an java.util.concurrent.ExecutorService, but for reasons of memory model consistency, this will be shut down by this class prior to completion.

Since
1.10

Field Summary

Modifier and TypeField and Description
private final ScatterGatherBackingStoreSupplier
backingStoreSupplier

private long
compressionDoneAt

private final int
compressionLevel

private final ExecutorService
executorService

private final Deque<Future<? extends ScatterZipOutputStream>>
futures

private long
scatterDoneAt

private final long
startedAt

private final Deque<ScatterZipOutputStream>
streams

private final ThreadLocal<ScatterZipOutputStream>
tlScatterStreams

Constructor Summary

AccessConstructor and Description
public
ParallelScatterZipCreator()

Constructs a ParallelScatterZipCreator with default threads, which is set to the number of available processors, as defined by Runtime#availableProcessors

public
ParallelScatterZipCreator(final ExecutorService
The executorService to use for parallel scheduling. For technical reasons, this will be shut down by this class.
executorService)

Constructs a ParallelScatterZipCreator

public
ParallelScatterZipCreator(final ExecutorService
The executorService to use. For technical reasons, this will be shut down by this class.
executorService, final ScatterGatherBackingStoreSupplier
The supplier of backing store which shall be used
backingStoreSupplier)

Constructs a ParallelScatterZipCreator

public
ParallelScatterZipCreator(final ExecutorService
The executorService to use. For technical reasons, this will be shut down by this class.
executorService, final ScatterGatherBackingStoreSupplier
The supplier of backing store which shall be used
backingStoreSupplier, final int
The compression level used in compression, this value should be -1(default level) or between 0~9.
compressionLevel)

Constructs a ParallelScatterZipCreator

Method Summary

Modifier and TypeMethod and Description
public void
addArchiveEntry(final ZipArchiveEntry
The entry to add.
zipArchiveEntry, final InputStreamSupplier
The source input stream supplier
source)

Adds an archive entry to this archive.

public void
addArchiveEntry(final ZipArchiveEntryRequestSupplier
Should supply the entry to be added.
zipArchiveEntryRequestSupplier)

Adds an archive entry to this archive.

private void
closeAll()

public final Callable<ScatterZipOutputStream>

Returns:

A callable that should subsequently be passed to #submitStreamAwareCallable, possibly in a wrapped/adapted from. The value of this callable is not used, but any exceptions happening inside the compression will be propagated through the callable.
createCallable(final ZipArchiveEntry
The entry to add.
zipArchiveEntry, final InputStreamSupplier
The source input stream supplier
source)

Creates a callable that will compress the given archive entry.

public final Callable<ScatterZipOutputStream>

Returns:

A callable that should subsequently be passed to #submitStreamAwareCallable, possibly in a wrapped/adapted from. The value of this callable is not used, but any exceptions happening inside the compression will be propagated through the callable.
createCallable(final ZipArchiveEntryRequestSupplier
Should supply the entry to be added.
zipArchiveEntryRequestSupplier)

Creates a callable that will compress archive entry supplied by ZipArchiveEntryRequestSupplier.

private ScatterZipOutputStream
createDeferred(final ScatterGatherBackingStoreSupplier scatterGatherBackingStoreSupplier)

public ScatterStatistics

Returns:

A string
getStatisticsMessage()

Gets a message describing the overall statistics of the compression run

public final void
submit(final Callable<? extends Object>
The callable to run, created by createCallable, possibly wrapped by caller.
callable)

Submits a callable for compression.

public final void
submitStreamAwareCallable(final Callable<? extends ScatterZipOutputStream>
The callable to run, created by createCallable, possibly wrapped by caller.
callable)

Submits a callable for compression.

public void
writeTo(final ZipArchiveOutputStream
The ZipArchiveOutputStream to receive the contents of the scatter streams
targetStream)

Writes the contents this to the target ZipArchiveOutputStream.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

backingStoreSupplierback to summary
private final ScatterGatherBackingStoreSupplier backingStoreSupplier
compressionDoneAtback to summary
private long compressionDoneAt
compressionLevelback to summary
private final int compressionLevel
executorServiceback to summary
private final ExecutorService executorService
futuresback to summary
private final Deque<Future<? extends ScatterZipOutputStream>> futures
scatterDoneAtback to summary
private long scatterDoneAt
startedAtback to summary
private final long startedAt
streamsback to summary
private final Deque<ScatterZipOutputStream> streams
tlScatterStreamsback to summary
private final ThreadLocal<ScatterZipOutputStream> tlScatterStreams

Constructor Detail

ParallelScatterZipCreatorback to summary
public ParallelScatterZipCreator()

Constructs a ParallelScatterZipCreator with default threads, which is set to the number of available processors, as defined by Runtime#availableProcessors

ParallelScatterZipCreatorback to summary
public ParallelScatterZipCreator(final ExecutorService executorService)

Constructs a ParallelScatterZipCreator

Parameters
executorService:ExecutorService

The executorService to use for parallel scheduling. For technical reasons, this will be shut down by this class.

ParallelScatterZipCreatorback to summary
public ParallelScatterZipCreator(final ExecutorService executorService, final ScatterGatherBackingStoreSupplier backingStoreSupplier)

Constructs a ParallelScatterZipCreator

Parameters
executorService:ExecutorService

The executorService to use. For technical reasons, this will be shut down by this class.

backingStoreSupplier:ScatterGatherBackingStoreSupplier

The supplier of backing store which shall be used

ParallelScatterZipCreatorback to summary
public ParallelScatterZipCreator(final ExecutorService executorService, final ScatterGatherBackingStoreSupplier backingStoreSupplier, final int compressionLevel) throws IllegalArgumentException

Constructs a ParallelScatterZipCreator

Parameters
executorService:ExecutorService

The executorService to use. For technical reasons, this will be shut down by this class.

backingStoreSupplier:ScatterGatherBackingStoreSupplier

The supplier of backing store which shall be used

compressionLevel:int

The compression level used in compression, this value should be -1(default level) or between 0~9.

Exceptions
IllegalArgumentException:
if the compression level is illegal
Since
1.21

Method Detail

addArchiveEntryback to summary
public void addArchiveEntry(final ZipArchiveEntry zipArchiveEntry, final InputStreamSupplier source)

Adds an archive entry to this archive.

This method is expected to be called from a single client thread

Parameters
zipArchiveEntry:ZipArchiveEntry

The entry to add.

source:InputStreamSupplier

The source input stream supplier

addArchiveEntryback to summary
public void addArchiveEntry(final ZipArchiveEntryRequestSupplier zipArchiveEntryRequestSupplier)

Adds an archive entry to this archive.

This method is expected to be called from a single client thread

Parameters
zipArchiveEntryRequestSupplier:ZipArchiveEntryRequestSupplier

Should supply the entry to be added.

Since
1.13
closeAllback to summary
private void closeAll()
createCallableback to summary
public final Callable<ScatterZipOutputStream> createCallable(final ZipArchiveEntry zipArchiveEntry, final InputStreamSupplier source)

Creates a callable that will compress the given archive entry.

This method is expected to be called from a single client thread.

Consider using addArchiveEntry, which wraps this method and submitStreamAwareCallable. The most common use case for using createCallable and submitStreamAwareCallable from a client is if you want to wrap the callable in something that can be prioritized by the supplied ExecutorService, for instance to process large or slow files first. Since the creation of the ExecutorService is handled by the client, all of this is up to the client.
Parameters
zipArchiveEntry:ZipArchiveEntry

The entry to add.

source:InputStreamSupplier

The source input stream supplier

Returns:Callable<ScatterZipOutputStream>

A callable that should subsequently be passed to #submitStreamAwareCallable, possibly in a wrapped/adapted from. The value of this callable is not used, but any exceptions happening inside the compression will be propagated through the callable.

createCallableback to summary
public final Callable<ScatterZipOutputStream> createCallable(final ZipArchiveEntryRequestSupplier zipArchiveEntryRequestSupplier)

Creates a callable that will compress archive entry supplied by ZipArchiveEntryRequestSupplier.

This method is expected to be called from a single client thread.

The same as createCallable(ZipArchiveEntry, InputStreamSupplier), but the archive entry to be added is supplied by a ZipArchiveEntryRequestSupplier.
Parameters
zipArchiveEntryRequestSupplier:ZipArchiveEntryRequestSupplier

Should supply the entry to be added.

Returns:Callable<ScatterZipOutputStream>

A callable that should subsequently be passed to #submitStreamAwareCallable, possibly in a wrapped/adapted from. The value of this callable is not used, but any exceptions happening inside the compression will be propagated through the callable.

Since
1.13
See Also
createCallable(ZipArchiveEntry, InputStreamSupplier)
createDeferredback to summary
private ScatterZipOutputStream createDeferred(final ScatterGatherBackingStoreSupplier scatterGatherBackingStoreSupplier) throws IOException
Annotations
@SuppressWarnings:resource
getStatisticsMessageback to summary
public ScatterStatistics getStatisticsMessage()

Gets a message describing the overall statistics of the compression run

Returns:ScatterStatistics

A string

submitback to summary
public final void submit(final Callable<? extends Object> callable)

Submits a callable for compression.

Parameters
callable:Callable<? extends Object>

The callable to run, created by createCallable, possibly wrapped by caller.

See Also
for details of if/when to use this.
submitStreamAwareCallableback to summary
public final void submitStreamAwareCallable(final Callable<? extends ScatterZipOutputStream> callable)

Submits a callable for compression.

Parameters
callable:Callable<? extends ScatterZipOutputStream>

The callable to run, created by createCallable, possibly wrapped by caller.

Since
1.19
See Also
for details of if/when to use this.
writeToback to summary
public void writeTo(final ZipArchiveOutputStream targetStream) throws IOException, InterruptedException, ExecutionException

Writes the contents this to the target ZipArchiveOutputStream.

It may be beneficial to write things like directories and manifest files to the targetStream before calling this method.

Calling this method will shut down the ExecutorService used by this class. If any of the Callables submitted to this instance throws an exception, the archive can not be created properly and this method will throw an exception.

Parameters
targetStream:ZipArchiveOutputStream

The ZipArchiveOutputStream to receive the contents of the scatter streams

Exceptions
IOException:
If writing fails
InterruptedException:
If we get interrupted
ExecutionException:
If something happens in the parallel execution