Top Description Fields Constructors Methods
org.eclipse.jgit.internal.storage.file

public Class ObjectDirectoryPackParser

extends PackParser
Class Inheritance
Imports
java.io.File, .FileOutputStream, .IOException, .InputStream, .RandomAccessFile, java.nio.file.StandardCopyOption, java.security.MessageDigest, java.text.MessageFormat, java.util.Arrays, .List, java.util.zip.CRC32, .Deflater, org.eclipse.jgit.errors.LockFailedException, org.eclipse.jgit.internal.JGitText, org.eclipse.jgit.internal.storage.pack.PackExt, .PackIndexWriter, org.eclipse.jgit.lib.AnyObjectId, .Constants, .CoreConfig, .ObjectId, .ProgressMonitor, org.eclipse.jgit.storage.pack.PackConfig, org.eclipse.jgit.transport.PackLock, .PackParser, .PackedObjectInfo, org.eclipse.jgit.util.FileUtils, .NB
Consumes a pack stream and stores as a pack file in org.eclipse.jgit.internal.storage.file.ObjectDirectory.

To obtain an instance of a parser, applications should use org.eclipse.jgit.lib.ObjectInserter#newPackParser(InputStream).

Field Summary

Modifier and TypeField and Description
private final CRC32
crc

CRC-32 computation for objects that are appended onto the pack.

private final FileObjectDatabase
db

private Deflater
def

Compresses delta bases when completing a thin pack.

private int
indexVersion

Preferred format version of the pack-*.idx file to generate.

private boolean
keepEmpty

If true, pack with 0 objects will be stored.

private Pack
newPack

The pack that was created, if parsing was successful.

private long
origEnd

Length of the original pack stream, before missing bases were appended.

private byte[]
origHash

The original checksum of data up to origEnd.

private RandomAccessFile
out

Read/write handle to tmpPack while it is being parsed.

private long
packEnd

Current end of the pack file.

private byte[]
packHash

Checksum of the entire pack file.

private PackConfig
pconfig

private final MessageDigest
tailDigest

Running SHA-1 of any base objects appended after origEnd.

private File
tmpIdx

Path of the index created for the pack, to find objects quickly at read time.

private File
tmpPack

Path of the temporary file holding the pack data.

Constructor Summary

AccessConstructor and Description
pack-priv
ObjectDirectoryPackParser(FileObjectDatabase odb, InputStream src)

Method Summary

Modifier and TypeMethod and Description
private static String
baseName(File tmpPack)

protected boolean
checkCRC(int
the prior CRC that was recorded during the first scan of the object from the pack stream.
oldCRC)

Implements abstract org.eclipse.jgit.transport.PackParser.checkCRC.

Check the current CRC matches the expected value.

private void
cleanupTemporaryFiles()

public Pack

Returns:

the imported PackFile, if parsing was successful.
getPack()

Get the imported org.eclipse.jgit.internal.storage.file.Pack.

public long
getPackSize()

Overrides org.eclipse.jgit.transport.PackParser.getPackSize.

Get the size of the newly created pack.

protected boolean
onAppendBase(final int
type of the base object.
typeCode, final byte[]
complete content of the base object.
data, final PackedObjectInfo
packed object information for this base. Implementors must populate the CRC and offset members if returning true.
info)

Implements abstract org.eclipse.jgit.transport.PackParser.onAppendBase.

Provide the implementation with a base that was outside of the pack.

protected void
onBeginOfsDelta(long
position of this object in the incoming stream.
streamPosition, long
position of the base object in the incoming stream. The base must be before the delta, therefore baseStreamPosition < deltaStreamPosition. This is not the position returned by a prior end object event.
baseStreamPosition, long
size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.
inflatedSize)

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginOfsDelta.

Event notifying start of a delta referencing its base by offset.

protected void
onBeginRefDelta(long
position of this object in the incoming stream.
streamPosition, AnyObjectId
name of the base object. This object may be later in the stream, or might not appear at all in the stream (in the case of a thin-pack).
baseId, long
size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.
inflatedSize)

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginRefDelta.

Event notifying start of a delta referencing its base by ObjectId.

protected void
onBeginWholeObject(long
position of this object in the incoming stream.
streamPosition, int
type of the object; one of org.eclipse.jgit.lib.Constants#OBJ_COMMIT, org.eclipse.jgit.lib.Constants#OBJ_TREE, org.eclipse.jgit.lib.Constants#OBJ_BLOB, or org.eclipse.jgit.lib.Constants#OBJ_TAG.
type, long
size of the object when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.
inflatedSize)

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginWholeObject.

Event notifying the start of an object stored whole (not as a delta).

protected PackParser.UnresolvedDelta
onEndDelta()

Overrides org.eclipse.jgit.transport.PackParser.onEndDelta.

Event notifying the current object.

protected void
onEndThinPack()

Implements abstract org.eclipse.jgit.transport.PackParser.onEndThinPack.

Event indicating a thin pack has been completely processed.

protected void
onEndWholeObject(PackedObjectInfo
object information.
info)

Implements abstract org.eclipse.jgit.transport.PackParser.onEndWholeObject.

Event notifying the current object.

protected void
onInflatedObjectData(PackedObjectInfo
the object info, populated.
obj, int
the type of the object.
typeCode, byte[]
inflated data for the object.
data)

Implements abstract org.eclipse.jgit.transport.PackParser.onInflatedObjectData.

Invoked for commits, trees, tags, and small blobs.

protected void
onObjectData(PackParser.Source
where the data came from
src, byte[]
buffer to read data from.
raw, int
first offset within buffer that is valid.
pos, int
number of bytes in buffer that are valid.
len)

Implements abstract org.eclipse.jgit.transport.PackParser.onObjectData.

Store (and/or checksum) a portion of an object's data.

protected void
onObjectHeader(PackParser.Source
where the data came from
src, byte[]
buffer to read data from.
raw, int
first offset within buffer that is valid.
pos, int
number of bytes in buffer that are valid.
len)

Implements abstract org.eclipse.jgit.transport.PackParser.onObjectHeader.

Store (and/or checksum) an object header.

protected void
onPackFooter(byte[]
the trailing 20 bytes of the pack, this is a SHA-1 checksum of all of the pack data.
hash)

Implements abstract org.eclipse.jgit.transport.PackParser.onPackFooter.

Provide the implementation with the original stream's pack footer.

protected void
onPackHeader(long
number of objects expected in the stream.
objectCount)

Implements abstract org.eclipse.jgit.transport.PackParser.onPackHeader.

Provide the implementation with the original stream's pack header.

protected void
onStoreStream(byte[]
buffer to copy data out of.
raw, int
first offset within the buffer that is valid.
pos, int
number of bytes in the buffer that are valid.
len)

Implements abstract org.eclipse.jgit.transport.PackParser.onStoreStream.

Store bytes received from the raw stream.

public PackLock
parse(ProgressMonitor
receives progress feedback during the initial receiving objects phase. If null, org.eclipse.jgit.lib.NullProgressMonitor will be used.
receiving, ProgressMonitor
receives progress feedback during the resolving objects phase.
resolving)

Overrides org.eclipse.jgit.transport.PackParser.parse.

Parse the pack stream.

protected int
readDatabase(byte[]
the buffer to copy read data into.
dst, int
position within dst to start copying data into.
pos, int
ideal target number of bytes to read. Actual read length may be shorter.
cnt)

Implements abstract org.eclipse.jgit.transport.PackParser.readDatabase.

Read from the database's current position into the buffer.

private PackLock
renameAndOpenPack(String lockMessage)

protected PackParser.ObjectTypeAndSize
seekDatabase(PackParser.UnresolvedDelta
the object position to begin reading from. This is an instance previously returned by onEndDelta().
delta, PackParser.ObjectTypeAndSize
object to populate with type and size.
info)

Implements abstract org.eclipse.jgit.transport.PackParser.seekDatabase.

Reposition the database to re-read a previously stored object.

protected PackParser.ObjectTypeAndSize
seekDatabase(PackedObjectInfo
the object position to begin reading from. This is from newInfo(AnyObjectId, UnresolvedDelta, ObjectId).
obj, PackParser.ObjectTypeAndSize
object to populate with type and size.
info)

Implements abstract org.eclipse.jgit.transport.PackParser.seekDatabase.

Reposition the database to re-read a previously stored object.

public void
setIndexVersion(int
the version to write. The special version 0 designates the oldest (most compatible) format available for the objects.
version)

Set the pack index file format version this instance will create.

public void
setKeepEmpty(boolean
true to enable keeping an empty pack.
empty)

Configure this index pack instance to keep an empty pack.

private void
writeIdx()

Inherited from org.eclipse.jgit.transport.PackParser:
buffergetBaseObjectIdsgetLockMessagegetNewObjectIdsgetObjectgetObjectCountgetReceivedPackStatisticsgetSortedObjectListisAllowThinisCheckEofAfterPackFooterisCheckObjectCollisionsisExpectDataAfterPackFooternewInfoparsereadObjectHeadersetAllowThinsetCheckEofAfterPackFootersetCheckObjectCollisionssetExpectDataAfterPackFootersetExpectedObjectCountsetLockMessagesetMaxObjectSizeLimitsetNeedBaseObjectIdssetNeedNewObjectIdssetObjectCheckersetObjectCheckingverifySafeObject

Field Detail

crcback to summary
private final CRC32 crc

CRC-32 computation for objects that are appended onto the pack.

dbback to summary
private final FileObjectDatabase db
defback to summary
private Deflater def

Compresses delta bases when completing a thin pack.

indexVersionback to summary
private int indexVersion

Preferred format version of the pack-*.idx file to generate.

keepEmptyback to summary
private boolean keepEmpty

If true, pack with 0 objects will be stored. Usually these are deleted.

newPackback to summary
private Pack newPack

The pack that was created, if parsing was successful.

origEndback to summary
private long origEnd

Length of the original pack stream, before missing bases were appended.

origHashback to summary
private byte[] origHash

The original checksum of data up to origEnd.

outback to summary
private RandomAccessFile out

Read/write handle to tmpPack while it is being parsed.

packEndback to summary
private long packEnd

Current end of the pack file.

packHashback to summary
private byte[] packHash

Checksum of the entire pack file.

pconfigback to summary
private PackConfig pconfig
tailDigestback to summary
private final MessageDigest tailDigest

Running SHA-1 of any base objects appended after origEnd.

tmpIdxback to summary
private File tmpIdx

Path of the index created for the pack, to find objects quickly at read time.

tmpPackback to summary
private File tmpPack

Path of the temporary file holding the pack data.

Constructor Detail

ObjectDirectoryPackParserback to summary
pack-priv ObjectDirectoryPackParser(FileObjectDatabase odb, InputStream src)

Method Detail

baseNameback to summary
private static String baseName(File tmpPack)
checkCRCback to summary
protected boolean checkCRC(int oldCRC)

Implements abstract org.eclipse.jgit.transport.PackParser.checkCRC.

Doc from org.eclipse.jgit.transport.PackParser.checkCRC.

Check the current CRC matches the expected value.

This method is invoked when an object is read back in from the database and its data is used during delta resolution. The CRC is validated after the object has been fully read, allowing the parser to verify there was no silent data corruption.

Implementations are free to ignore this check by always returning true if they are performing other data integrity validations at a lower level.

Parameters
oldCRC:int

the prior CRC that was recorded during the first scan of the object from the pack stream.

Returns:boolean

true if the CRC matches; false if it does not.

Annotations
@Override
cleanupTemporaryFilesback to summary
private void cleanupTemporaryFiles()
getPackback to summary
public Pack getPack()

Get the imported org.eclipse.jgit.internal.storage.file.Pack.

This method is supplied only to support testing; applications shouldn't be using it directly to access the imported data.

Returns:Pack

the imported PackFile, if parsing was successful.

getPackSizeback to summary
public long getPackSize()

Overrides org.eclipse.jgit.transport.PackParser.getPackSize.

Doc from org.eclipse.jgit.transport.PackParser.getPackSize.

Get the size of the newly created pack.

This will also include the pack index size if an index was created. This method should only be called after pack parsing is finished.

Returns:long

the pack size (including the index size) or -1 if the size cannot be determined

Annotations
@Override
onAppendBaseback to summary
protected boolean onAppendBase(final int typeCode, final byte[] data, final PackedObjectInfo info) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onAppendBase.

Doc from org.eclipse.jgit.transport.PackParser.onAppendBase.

Provide the implementation with a base that was outside of the pack.

This event only occurs on a thin pack for base objects that were outside of the pack and came from the local repository. Usually an implementation uses this event to compress the base and append it onto the end of the pack, so the pack stays self-contained.

Parameters
typeCode:int

type of the base object.

data:byte[]

complete content of the base object.

info:PackedObjectInfo

packed object information for this base. Implementors must populate the CRC and offset members if returning true.

Returns:boolean

true if the info should be included in the object list returned by getSortedObjectList(Comparator), false if it should not be included.

Annotations
@Override
Exceptions
IOException:
the base could not be included into the pack.
onBeginOfsDeltaback to summary
protected void onBeginOfsDelta(long streamPosition, long baseStreamPosition, long inflatedSize) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginOfsDelta.

Doc from org.eclipse.jgit.transport.PackParser.onBeginOfsDelta.

Event notifying start of a delta referencing its base by offset.

Parameters
streamPosition:long

position of this object in the incoming stream.

baseStreamPosition:long

position of the base object in the incoming stream. The base must be before the delta, therefore baseStreamPosition < deltaStreamPosition. This is not the position returned by a prior end object event.

inflatedSize:long

size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onBeginRefDeltaback to summary
protected void onBeginRefDelta(long streamPosition, AnyObjectId baseId, long inflatedSize) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginRefDelta.

Doc from org.eclipse.jgit.transport.PackParser.onBeginRefDelta.

Event notifying start of a delta referencing its base by ObjectId.

Parameters
streamPosition:long

position of this object in the incoming stream.

baseId:AnyObjectId

name of the base object. This object may be later in the stream, or might not appear at all in the stream (in the case of a thin-pack).

inflatedSize:long

size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onBeginWholeObjectback to summary
protected void onBeginWholeObject(long streamPosition, int type, long inflatedSize) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginWholeObject.

Doc from org.eclipse.jgit.transport.PackParser.onBeginWholeObject.

Event notifying the start of an object stored whole (not as a delta).

Parameters
streamPosition:long

position of this object in the incoming stream.

type:int

type of the object; one of org.eclipse.jgit.lib.Constants#OBJ_COMMIT, org.eclipse.jgit.lib.Constants#OBJ_TREE, org.eclipse.jgit.lib.Constants#OBJ_BLOB, or org.eclipse.jgit.lib.Constants#OBJ_TAG.

inflatedSize:long

size of the object when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onEndDeltaback to summary
protected PackParser.UnresolvedDelta onEndDelta() throws IOException

Overrides org.eclipse.jgit.transport.PackParser.onEndDelta.

Doc from org.eclipse.jgit.transport.PackParser.onEndDelta.

Event notifying the current object.

Returns:PackParser.UnresolvedDelta

object information that must be populated with at least the offset.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onEndThinPackback to summary
protected void onEndThinPack() throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onEndThinPack.

Doc from org.eclipse.jgit.transport.PackParser.onEndThinPack.

Event indicating a thin pack has been completely processed.

This event is invoked only if a thin pack has delta references to objects external from the pack. The event is called after all of those deltas have been resolved.

Annotations
@Override
Exceptions
IOException:
the pack cannot be archived.
onEndWholeObjectback to summary
protected void onEndWholeObject(PackedObjectInfo info) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onEndWholeObject.

Doc from org.eclipse.jgit.transport.PackParser.onEndWholeObject.

Event notifying the current object.

Parameters
info:PackedObjectInfo

object information.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onInflatedObjectDataback to summary
protected void onInflatedObjectData(PackedObjectInfo obj, int typeCode, byte[] data) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onInflatedObjectData.

Doc from org.eclipse.jgit.transport.PackParser.onInflatedObjectData.

Invoked for commits, trees, tags, and small blobs.

Parameters
obj:PackedObjectInfo

the object info, populated.

typeCode:int

the type of the object.

data:byte[]

inflated data for the object.

Annotations
@Override
Exceptions
IOException:
the object cannot be archived.
onObjectDataback to summary
protected void onObjectData(PackParser.Source src, byte[] raw, int pos, int len) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onObjectData.

Doc from org.eclipse.jgit.transport.PackParser.onObjectData.

Store (and/or checksum) a portion of an object's data.

This method may be invoked multiple times per object, depending on the size of the object, the size of the parser's internal read buffer, and the alignment of the object relative to the read buffer.

Invoked after onObjectHeader(Source, byte[], int, int).

Parameters
src:PackParser.Source

where the data came from

raw:byte[]

buffer to read data from.

pos:int

first offset within buffer that is valid.

len:int

number of bytes in buffer that are valid.

Annotations
@Override
Exceptions
IOException:
the stream cannot be archived.
onObjectHeaderback to summary
protected void onObjectHeader(PackParser.Source src, byte[] raw, int pos, int len) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onObjectHeader.

Doc from org.eclipse.jgit.transport.PackParser.onObjectHeader.

Store (and/or checksum) an object header.

Invoked after any of the onBegin() events. The entire header is supplied in a single invocation, before any object data is supplied.

Parameters
src:PackParser.Source

where the data came from

raw:byte[]

buffer to read data from.

pos:int

first offset within buffer that is valid.

len:int

number of bytes in buffer that are valid.

Annotations
@Override
Exceptions
IOException:
the stream cannot be archived.
onPackFooterback to summary
protected void onPackFooter(byte[] hash) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onPackFooter.

Doc from org.eclipse.jgit.transport.PackParser.onPackFooter.

Provide the implementation with the original stream's pack footer.

Parameters
hash:byte[]

the trailing 20 bytes of the pack, this is a SHA-1 checksum of all of the pack data.

Annotations
@Override
Exceptions
IOException:
the stream cannot be archived.
onPackHeaderback to summary
protected void onPackHeader(long objectCount) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onPackHeader.

Doc from org.eclipse.jgit.transport.PackParser.onPackHeader.

Provide the implementation with the original stream's pack header.

Parameters
objectCount:long

number of objects expected in the stream.

Annotations
@Override
Exceptions
IOException:
the implementation refuses to work with this many objects.
onStoreStreamback to summary
protected void onStoreStream(byte[] raw, int pos, int len) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onStoreStream.

Doc from org.eclipse.jgit.transport.PackParser.onStoreStream.

Store bytes received from the raw stream.

This method is invoked during parse(ProgressMonitor) as data is consumed from the incoming stream. Implementors may use this event to archive the raw incoming stream to the destination repository in large chunks, without paying attention to object boundaries.

The only component of the pack not supplied to this method is the last 20 bytes of the pack that comprise the trailing SHA-1 checksum. Those are passed to onPackFooter(byte[]).

Parameters
raw:byte[]

buffer to copy data out of.

pos:int

first offset within the buffer that is valid.

len:int

number of bytes in the buffer that are valid.

Annotations
@Override
Exceptions
IOException:
the stream cannot be archived.
parseback to summary
public PackLock parse(ProgressMonitor receiving, ProgressMonitor resolving) throws IOException

Overrides org.eclipse.jgit.transport.PackParser.parse.

Doc from org.eclipse.jgit.transport.PackParser.parse.

Parse the pack stream.

Parameters
receiving:ProgressMonitor

receives progress feedback during the initial receiving objects phase. If null, org.eclipse.jgit.lib.NullProgressMonitor will be used.

resolving:ProgressMonitor

receives progress feedback during the resolving objects phase.

Returns:PackLock

the pack lock, if one was requested by setting setLockMessage(String).

Annotations
@Override
Exceptions
IOException:
the stream is malformed, or contains corrupt objects.
readDatabaseback to summary
protected int readDatabase(byte[] dst, int pos, int cnt) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.readDatabase.

Doc from org.eclipse.jgit.transport.PackParser.readDatabase.

Read from the database's current position into the buffer.

Parameters
dst:byte[]

the buffer to copy read data into.

pos:int

position within dst to start copying data into.

cnt:int

ideal target number of bytes to read. Actual read length may be shorter.

Returns:int

number of bytes stored.

Annotations
@Override
Exceptions
IOException:
the database cannot be accessed.
renameAndOpenPackback to summary
private PackLock renameAndOpenPack(String lockMessage) throws IOException
seekDatabaseback to summary
protected PackParser.ObjectTypeAndSize seekDatabase(PackParser.UnresolvedDelta delta, PackParser.ObjectTypeAndSize info) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.seekDatabase.

Doc from org.eclipse.jgit.transport.PackParser.seekDatabase.

Reposition the database to re-read a previously stored object.

If the database is computing CRC-32 checksums for object data, it should reset its internal CRC instance during this method call.

Parameters
delta:PackParser.UnresolvedDelta

the object position to begin reading from. This is an instance previously returned by onEndDelta().

info:PackParser.ObjectTypeAndSize

object to populate with type and size.

Returns:PackParser.ObjectTypeAndSize

the info object.

Annotations
@Override
Exceptions
IOException:
the database cannot reposition to this location.
seekDatabaseback to summary
protected PackParser.ObjectTypeAndSize seekDatabase(PackedObjectInfo obj, PackParser.ObjectTypeAndSize info) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.seekDatabase.

Doc from org.eclipse.jgit.transport.PackParser.seekDatabase.

Reposition the database to re-read a previously stored object.

If the database is computing CRC-32 checksums for object data, it should reset its internal CRC instance during this method call.

Parameters
obj:PackedObjectInfo

the object position to begin reading from. This is from newInfo(AnyObjectId, UnresolvedDelta, ObjectId).

info:PackParser.ObjectTypeAndSize

object to populate with type and size.

Returns:PackParser.ObjectTypeAndSize

the info object.

Annotations
@Override
Exceptions
IOException:
the database cannot reposition to this location.
setIndexVersionback to summary
public void setIndexVersion(int version)

Set the pack index file format version this instance will create.

Parameters
version:int

the version to write. The special version 0 designates the oldest (most compatible) format available for the objects.

See Also
BasePackIndexWriter
setKeepEmptyback to summary
public void setKeepEmpty(boolean empty)

Configure this index pack instance to keep an empty pack.

By default an empty pack (a pack with no objects) is not kept, as doi so is completely pointless. With no objects in the pack there is no d stored by it, so the pack is unnecessary.

Parameters
empty:boolean

true to enable keeping an empty pack.

writeIdxback to summary
private void writeIdx() throws IOException