Top Description Inners Fields Constructors Methods
org.eclipse.jgit.internal.storage.pack

public Class PackWriter

extends Object
implements AutoCloseable
Class Inheritance
All Implemented Interfaces
java.lang.AutoCloseable
Static Imports
java.util.Objects.requireNonNull, org.eclipse.jgit.internal.storage.pack.StoredObjectRepresentation.PACK_DELTA, .StoredObjectRepresentation.PACK_WHOLE, org.eclipse.jgit.lib.Constants.OBJECT_ID_LENGTH, .Constants.OBJ_BLOB, .Constants.OBJ_COMMIT, .Constants.OBJ_TAG, .Constants.OBJ_TREE

PackWriter class is responsible for generating pack files from specified set of objects from repository. This implementation produce pack files in format version 2.

Source of objects may be specified in two ways:

Typical usage consists of creating an instance, configuring options, preparing the list of objects by calling preparePack(Iterator) or preparePack(ProgressMonitor, Set, Set), and streaming with writePack(ProgressMonitor, ProgressMonitor, OutputStream). If the pack is being stored as a file the matching index can be written out after writing the pack by writeIndex(PackIndexWriter). An optional bitmap index can be made by calling prepareBitmapIndex(ProgressMonitor) followed by writeBitmapIndex(PackBitmapIndexWriter).

Class provide set of configurable options and org.eclipse.jgit.lib.ProgressMonitor support, as operations may take a long time for big repositories. Deltas searching algorithm is NOT IMPLEMENTED yet - this implementation relies only on deltas and objects reuse.

This class is not thread safe. It is intended to be used in one thread as a single pass to produce one pack. Invoking methods multiple times or out of order is not supported as internal data structures are destroyed during certain phases to save memory when packing large repositories.

Nested and Inner Type Summary

Modifier and TypeClass and Description
private static class
PackWriter.DepthAwareVisitationPolicy

A visitation policy which uses the depth at which the object is seen to decide if re-traversal is necessary.

private class
public static class
PackWriter.PackfileUriConfig

Configuration related to the packfile URI feature.

public static enum
PackWriter.PackingPhase

Possible states that a PackWriter can be in.

public class
PackWriter.State

Summary of the current state of a PackWriter.

Field Summary

Modifier and TypeField and Description
private List<CachedPack>
private ObjectCountCallback
private boolean
private boolean
pack-priv final PackConfig
private CRC32
private boolean
private boolean
private int
private List<ObjectToPack>
private Set<? extends ObjectId>
private ObjectIdSet
private ObjectIdSet[]
private FilterSpec
private BitmapIndex.BitmapBuilder
private boolean
private boolean
private static final Map<WeakReference<PackWriter>, Boolean>
private Deflater
public static final Set<ObjectId>
NONE

Empty set of objects for preparePack().

pack-priv BlockList<ObjectToPack>[]
private ObjectIdOwnerMap<ObjectToPack>
private static final int
private byte[]
private PackWriter.PackfileUriConfig
private boolean
private final ObjectReader
private boolean
private boolean
private final ObjectReuseAsIs
reuseSupport

reader recast to the reuse interface, if it supports it.

private boolean
private long
private final Duration
private final WeakReference<PackWriter>
private boolean
private List<ObjectToPack>
private final PackWriter.MutableState
private final PackStatistics.Accumulator
private Set<ObjectId>
private boolean
private PackStatistics.ObjectType.Accumulator
private Collection<? extends ObjectId>
private boolean
private boolean
private PackBitmapIndexBuilder

Constructor Summary

AccessConstructor and Description
public
PackWriter(Repository
repository where objects are stored.
repo
)

Create writer for specified repository.

public
PackWriter(Repository
repository where objects are stored.
repo
,
ObjectReader
reader to read from the repository with.
reader
)

Create writer for specified repository.

public
PackWriter(PackConfig
configuration for the pack writer.
config
,
ObjectReader
reader to read from the repository with.
reader
)

Create writer with a specified configuration.

public
PackWriter(PackConfig
configuration for the pack writer.
config
,
final ObjectReader
reader to read from the repository with.
reader
,
PackStatistics.Accumulator
accumulator for statics
statsAccumulator
)

Create writer with a specified configuration.

Method Summary

Modifier and TypeMethod and Description
public void
addObject(RevObject
the object to add.
object
)

Include one object to the output file.

private void
addObject(RevObject object, int pathHashCode)

private void
addObject(final AnyObjectId src, final int type, final int pathHashCode)

private void
private byte[]
pack-priv static byte[]
public void
checkSearchForReuseTimeout()

Check whether the search for reuse phase is taking too long.

public void
close()

Implements java.lang.AutoCloseable.close.

Closes this resource, relinquishing any underlying resources.

public ObjectId

Returns:

ObjectId representing SHA-1 name of a pack that was created.
computeName
()

Computes SHA-1 of lexicographically sorted objects ids written in this pack, as used to name a pack file in repository.

private void
private Deflater
private TemporaryBuffer.Heap
private boolean

Returns:

whether the given object should be skipped.
depthSkip
(RevObject
the object to check whether it should be omitted.
obj
,
ObjectWalk
the walker being used for traveresal.
walker
)

Determines if the object should be omitted from the pack as a result of its depth (probably because of the tree:<depth> filter).

public void
enableSearchForReuseTimeout()

Set the writer to check for long search for reuse, exceeding the timeout.

private void
private boolean
exclude(AnyObjectId objectId)

public void
excludeObjects(ObjectIdSet
objects in this index will not be in the output pack.
idx
)

Add a pack index whose contents should be excluded from the result.

private void
filterAndAddObject(AnyObjectId src, int type, int pathHashCode, Set<? extends AnyObjectId> want)

private int
findObjectsNeedingDelta(ObjectToPack[] list, int cnt, int type)

private void
findObjectsToPack(ProgressMonitor countingMonitor, ObjectWalk walker, Set<? extends ObjectId> want, Set<? extends ObjectId> have, Set<? extends ObjectId> noBitmaps)

private void
findObjectsToPackUsingBitmaps(BitmapWalker bitmapWalker, Set<? extends ObjectId> want, Set<? extends ObjectId> have)

public ObjectToPack

Returns:

the object we are packing, or null.
get
(AnyObjectId
the object to find in the pack.
id
)

Lookup the ObjectToPack object for a given ObjectId.

public int

Returns:

the index format version.
getIndexVersion
()

Returns the index format version that will be written.

public static Iterable<PackWriter>

Returns:

all allocated, non-released PackWriters instances.
getInstances
()

Get all allocated, non-released PackWriters instances.

public long

Returns:

number of objects in pack.
getObjectCount
()

Returns objects number in a pack file that was created by this writer.

public ObjectIdOwnerMap<ObjectIdOwnerMap.Entry>

Returns:

set of objects in pack.
getObjectSet
()

Returns the object ids in the pack file that was created by this writer.

private ObjectWalk
public PackWriter.State

Returns:

snapshot of the current state of this PackWriter.
getState
()

Get snapshot of the current state of this PackWriter.

public PackStatistics

Returns:

description of what this PackWriter did in order to create the final pack stream. This should only be invoked after the calls to create the pack/index/bitmap have completed.
getStatistics
()

Get statistics of what this PackWriter did in order to create the final pack stream.

private long
private final boolean
have(ObjectToPack ptr, AnyObjectId objectId)

private static Iterator<PackWriter>
public boolean

Returns:

true if the bitmap file is to be created by this PackWriter.
isCreateBitmaps
()

Whether the bitmap file is to be created by this PackWriter.

public boolean

Returns:

true if delta base is stored as an offset; false if it is stored as an object id.
isDeltaBaseAsOffset
()

Check whether writer can store delta base as an offset (new style reducing pack size) or should store it as an object id (legacy style, compatible with old readers).

public boolean

Returns:

true to ignore objects that are uninteresting and also not found on local disk; false to throw a org.eclipse.jgit.errors.MissingObjectException out of preparePack(ProgressMonitor, Set, Set) if an uninteresting object is not in the source repository. By default, true, permitting gracefully ignoring of uninteresting objects.
isIgnoreMissingUninteresting
()

Whether to ignore missing uninteresting objects

public boolean

Returns:

true if the index file cannot be created by this PackWriter.
isIndexDisabled
()

Whether the index file cannot be created by this PackWriter.

public boolean

Returns:

true if the writer would reuse commits stored as deltas, assuming delta reuse is already enabled.
isReuseDeltaCommits
()

Check if the writer will reuse commits that are already stored as deltas.

public boolean

Returns:

true if validation is enabled; false if the reader will handle object validation as a side-effect of it consuming the output.
isReuseValidatingObjects
()

Check if the writer validates objects before copying them.

public boolean

Returns:

whether the writer will write a reverse index file
isReverseIndexEnabled
()

Whether the writer will write a reverse index file.

public boolean

Returns:

true if this writer is producing a thin pack.
isThin
()

Whether this writer is producing a thin pack.

public boolean

Returns:

true to use bitmaps for ObjectWalks, if available.
isUseBitmaps
()

Whether to use bitmaps

public boolean

Returns:

true to reuse cached packs. If true index creation isn't available.
isUseCachedPacks
()

Whether to reuse cached packs.

private void
parallelDeltaSearch(ProgressMonitor monitor, ObjectToPack[] list, int cnt, int threads)

public boolean

Returns:

whether a bitmap index may be written.
prepareBitmapIndex
(ProgressMonitor
progress monitor to report bitmap building work.
pm
)

Prepares the bitmaps to be written to the bitmap index file.

public void
preparePack(Iterator<RevObject>
iterator of object to store in a pack; order of objects within each type is important, ordering by type is not needed; allowed types for objects are org.eclipse.jgit.lib.Constants#OBJ_COMMIT, org.eclipse.jgit.lib.Constants#OBJ_TREE, org.eclipse.jgit.lib.Constants#OBJ_BLOB and org.eclipse.jgit.lib.Constants#OBJ_TAG; objects returned by iterator may be later reused by caller as object id and type are internally copied in each iteration.
objectsSource
)

Prepare the list of objects to be written to the pack stream.

public void
preparePack(Collection<? extends CachedPack>
cached packs to be written.
c
)

Prepare the list of objects to be written to the pack stream.

public void
preparePack(ProgressMonitor
progress during object enumeration.
countingMonitor
,
Set<? extends ObjectId>
collection of objects to be marked as interesting (start points of graph traversal). Must not be null.
want
,
Set<? extends ObjectId>
collection of objects to be marked as uninteresting (end points of graph traversal). Pass NONE if all objects reachable from want are desired, such as when serving a clone.
have
)

Prepare the list of objects to be written to the pack stream.

public void
preparePack(ProgressMonitor
progress during object enumeration.
countingMonitor
,
Set<? extends ObjectId>
objects of interest, ancestors of which will be included in the pack. Must not be null.
want
,
Set<? extends ObjectId>
objects whose ancestors (up to and including shallow commits) do not need to be included in the pack because they are already available from elsewhere. Must not be null.
have
,
Set<? extends ObjectId>
commits indicating the boundary of the history marked with have. Shallow commits have parents but those parents are considered not to be already available. Parents of shallow commits and earlier generations will be included in the pack if requested by want. Must not be null.
shallow
)

Prepare the list of objects to be written to the pack stream.

public void
preparePack(ProgressMonitor
progress during object enumeration.
countingMonitor
,
Set<? extends ObjectId>
objects of interest, ancestors of which will be included in the pack. Must not be null.
want
,
Set<? extends ObjectId>
objects whose ancestors (up to and including shallow commits) do not need to be included in the pack because they are already available from elsewhere. Must not be null.
have
,
Set<? extends ObjectId>
commits indicating the boundary of the history marked with have. Shallow commits have parents but those parents are considered not to be already available. Parents of shallow commits and earlier generations will be included in the pack if requested by want. Must not be null.
shallow
,
Set<? extends ObjectId>
collection of objects to be excluded from bitmap commit selection.
noBitmaps
)

Prepare the list of objects to be written to the pack stream.

public void
preparePack(ProgressMonitor
progress during object enumeration.
countingMonitor
,
ObjectWalk
ObjectWalk to perform enumeration.
walk
,
Set<? extends ObjectId>
collection of objects to be marked as interesting (start points of graph traversal). Must not be null.
interestingObjects
,
Set<? extends ObjectId>
collection of objects to be marked as uninteresting (end points of graph traversal). Pass NONE if all objects reachable from want are desired, such as when serving a clone.
uninterestingObjects
,
Set<? extends ObjectId>
collection of objects to be excluded from bitmap commit selection.
noBitmaps
)

Prepare the list of objects to be written to the pack stream.

private static void
private void
private boolean
private static void
private void
private void
searchForDeltas(final ProgressMonitor monitor, final ObjectToPack[] list, final int cnt)

private void
private void
public void
select(ObjectToPack
the object being packed.
otp
,
StoredObjectRepresentation
the next available representation from the repository.
next
)

Select an object representation for this writer.

public void
setClientShallowCommits(Set<ObjectId>
the shallow commits in the client
clientShallowCommits
)

Records the set of shallow commits in the client.

public void
setCreateBitmaps(boolean
if set to true, bitmaps will be generated when creating a pack.
createBitmaps
)

Whether to generate bitmaps.

public void
setDeltaBaseAsOffset(boolean
boolean indicating whether delta base can be stored as an offset.
deltaBaseAsOffset
)

Set writer delta base format.

public void
setFilterSpec(FilterSpec
the filter which indicates what and what not this writer should include
filter
)

Set filter spec

public void
setIgnoreMissingUninteresting(boolean
true if writer should ignore non existing uninteresting objects during construction set of objects to pack; false otherwise - non existing uninteresting objects may cause org.eclipse.jgit.errors.MissingObjectException
ignore
)

Whether writer should ignore non existing uninteresting objects

public void
setIndexDisabled(boolean
true to disable creation of the index file.
noIndex
)

Whether to disable creation of the index file.

public PackWriter

Returns:

this object for chaining.
setObjectCountCallback
(ObjectCountCallback
the callback to set
callback
)

Set the ObjectCountCallback.

public void
setPackfileUriConfig(PackWriter.PackfileUriConfig
configuration related to packfile URIs
config
)

Set packfile URI config

public void
setReuseDeltaCommits(boolean
if true, the writer will reuse any commits stored as deltas. By default the writer does not reuse delta commits.
reuse
)

Set the writer to reuse existing delta versions of commits.

public void
setReuseValidatingObjects(boolean
if true the pack writer will validate an object before it is put into the output. This additional validation work may be necessary to avoid propagating corruption from one local pack file to another local pack file.
validate
)

Enable (or disable) object validation during packing.

public void
setShallowPack(int
maximum depth of history to return. 1 means return only the "wants".
depth
,
Collection<? extends ObjectId>
objects which used to be shallow on the client, but are being extended as part of this fetch
unshallow
)

Configure this pack for a shallow clone.

public void
setTagTargets(Set<ObjectId>
objects that annotated tags point at.
objects
)

Set the tag targets that should be hoisted earlier during packing.

public void
setThin(boolean
a boolean indicating whether writer may pack objects with delta base object not within set of objects to pack, but belonging to party repository (uninteresting/boundary) as determined by set; this kind of pack is used only for transport; true - to produce thin pack, false - otherwise.
packthin
)

Whether writer may pack objects with delta base object not within set of objects to pack

public void
setUseBitmaps(boolean
if set to true, bitmaps will be used when preparing a pack.
useBitmaps
)

Whether to use bitmaps

public void
setUseCachedPacks(boolean
if set to true and a cached pack is present, it will be appended onto the end of a thin-pack, reducing the amount of working set space and CPU used by PackWriter. Enabling this feature prevents PackWriter from creating an index for the newly created pack, so its only suitable for writing to a network client, where the client will make the index.
useCached
)

Whether to use cached packs

private void
private List<ObjectToPack>
public boolean

Returns:

true if the object will appear in the output pack file.
willInclude
(AnyObjectId
the object to test the existence of.
id
)

Determine if the pack file will contain the requested object.

private void
public void
writeBitmapIndex(PackBitmapIndexWriter
a writer to store the bitmap index in this object database
bitmapIndexWriter
)

Create a bitmap index file to match the pack file just written.

private void
private void
public void
writeIndex(OutputStream
output for the index data. Caller is responsible for closing this stream.
indexStream
)

Create an index file to match the pack file just written.

public void
writeIndex(PackIndexWriter
an PackIndexWriter instance to write the index
iw
)

Create an index file to match the pack file just written.

pack-priv void
private void
private void
private void
public void
writeObjectSizeIndex(OutputStream
output for the object size index data. Caller is responsible for closing this stream.
objIdxStream
)

Create an object size index file for the contents of the pack file just written.

public void
writePack(ProgressMonitor
progress monitor to report object compression work.
compressMonitor
,
ProgressMonitor
progress monitor to report the number of objects written.
writeMonitor
,
OutputStream
output stream of pack data. The stream should be buffered by the caller. The caller is responsible for closing the stream.
packStream
)

Write the prepared pack to the supplied stream.

public void
writeReverseIndex(OutputStream
where to write the file contents to
stream
)

Write the pack's reverse index file to the output stream.

private void
Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait