Top Description Fields Constructors Methods
org.eclipse.jgit.internal.storage.dfs

public Class DfsPackParser

extends PackParser
Class Inheritance
Static Imports
org.eclipse.jgit.internal.storage.pack.PackExt.PACK
Parses a pack stream into the DFS, by creating a new pack and index.

Field Summary

Modifier and TypeField and Description
private DfsBlockCache
blockCache

Cache the chunks were stored into or get read back from.

private int
blockSize

Block size to use when caching data for read back.

private final CRC32
crc

CRC-32 computation for objects that are appended onto the pack.

private byte[]
currBuf

Data being written that has not yet been cached.

private int
currEnd

private long
currPos

private Deflater
def

Compresses delta bases when completing a thin pack.

private boolean
isEmptyPack

True if the pack is an empty pack.

private final DfsObjDatabase
objdb

private final DfsInserter
objins

private DfsOutputStream
out

Stream to the DFS storage, opened during onPackHeader(long).

private final MessageDigest
packDigest

Hides org.eclipse.jgit.transport.PackParser.packDigest.

Running SHA-1 of the entire pack stream.
private DfsPackDescription
packDsc

Name of the pack file, computed in onPackHeader(long).

private long
packEnd

Current end of the pack file.

private byte[]
packHash

Checksum of the entire pack file.

private PackIndex
packIndex

If the index was small enough, the entire index after writing.

private DfsStreamKey
packKey

Key used during delta resolution reading delta chains.

private DfsBlock
readBlock

private long
readPos

Cached block that is being read.

Constructor Summary

AccessConstructor and Description
protected
DfsPackParser(DfsObjDatabase
database the objects will be imported into.
db, DfsInserter
inserter the parser will use to help it inject the objects.
ins, InputStream
the stream to parse.
in)

Initialize a new pack parser.

Method Summary

Modifier and TypeMethod and Description
private void
buffer(byte[] raw, int pos, int len)

protected boolean
checkCRC(int
the prior CRC that was recorded during the first scan of the object from the pack stream.
oldCRC)

Implements abstract org.eclipse.jgit.transport.PackParser.checkCRC.

Check the current CRC matches the expected value.

private DfsBlock
flushBlock()

public DfsPackDescription

Returns:

description of the imported pack, if one was made.
getPackDescription()

Get description of the imported pack, if one was made.

protected boolean
onAppendBase(final int
type of the base object.
typeCode, final byte[]
complete content of the base object.
data, final PackedObjectInfo
packed object information for this base. Implementors must populate the CRC and offset members if returning true.
info)

Implements abstract org.eclipse.jgit.transport.PackParser.onAppendBase.

Provide the implementation with a base that was outside of the pack.

protected void
onBeginOfsDelta(long
position of this object in the incoming stream.
streamPosition, long
position of the base object in the incoming stream. The base must be before the delta, therefore baseStreamPosition < deltaStreamPosition. This is not the position returned by a prior end object event.
baseStreamPosition, long
size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.
inflatedSize)

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginOfsDelta.

Event notifying start of a delta referencing its base by offset.

protected void
onBeginRefDelta(long
position of this object in the incoming stream.
streamPosition, AnyObjectId
name of the base object. This object may be later in the stream, or might not appear at all in the stream (in the case of a thin-pack).
baseId, long
size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.
inflatedSize)

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginRefDelta.

Event notifying start of a delta referencing its base by ObjectId.

protected void
onBeginWholeObject(long
position of this object in the incoming stream.
streamPosition, int
type of the object; one of org.eclipse.jgit.lib.Constants#OBJ_COMMIT, org.eclipse.jgit.lib.Constants#OBJ_TREE, org.eclipse.jgit.lib.Constants#OBJ_BLOB, or org.eclipse.jgit.lib.Constants#OBJ_TAG.
type, long
size of the object when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.
inflatedSize)

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginWholeObject.

Event notifying the start of an object stored whole (not as a delta).

protected PackParser.UnresolvedDelta
onEndDelta()

Overrides org.eclipse.jgit.transport.PackParser.onEndDelta.

Event notifying the current object.

protected void
onEndThinPack()

Implements abstract org.eclipse.jgit.transport.PackParser.onEndThinPack.

Event indicating a thin pack has been completely processed.

protected void
onEndWholeObject(PackedObjectInfo
object information.
info)

Implements abstract org.eclipse.jgit.transport.PackParser.onEndWholeObject.

Event notifying the current object.

protected void
onInflatedObjectData(PackedObjectInfo
the object info, populated.
obj, int
the type of the object.
typeCode, byte[]
inflated data for the object.
data)

Implements abstract org.eclipse.jgit.transport.PackParser.onInflatedObjectData.

Invoked for commits, trees, tags, and small blobs.

protected void
onObjectData(PackParser.Source
where the data came from
src, byte[]
buffer to read data from.
raw, int
first offset within buffer that is valid.
pos, int
number of bytes in buffer that are valid.
len)

Implements abstract org.eclipse.jgit.transport.PackParser.onObjectData.

Store (and/or checksum) a portion of an object's data.

protected void
onObjectHeader(PackParser.Source
where the data came from
src, byte[]
buffer to read data from.
raw, int
first offset within buffer that is valid.
pos, int
number of bytes in buffer that are valid.
len)

Implements abstract org.eclipse.jgit.transport.PackParser.onObjectHeader.

Store (and/or checksum) an object header.

protected void
onPackFooter(byte[]
the trailing 20 bytes of the pack, this is a SHA-1 checksum of all of the pack data.
hash)

Implements abstract org.eclipse.jgit.transport.PackParser.onPackFooter.

Provide the implementation with the original stream's pack footer.

protected void
onPackHeader(long
number of objects expected in the stream.
objectCount)

Implements abstract org.eclipse.jgit.transport.PackParser.onPackHeader.

Provide the implementation with the original stream's pack header.

protected void
onStoreStream(byte[]
buffer to copy data out of.
raw, int
first offset within the buffer that is valid.
pos, int
number of bytes in the buffer that are valid.
len)

Implements abstract org.eclipse.jgit.transport.PackParser.onStoreStream.

Store bytes received from the raw stream.

public PackLock
parse(ProgressMonitor
receives progress feedback during the initial receiving objects phase. If null, org.eclipse.jgit.lib.NullProgressMonitor will be used.
receiving, ProgressMonitor
receives progress feedback during the resolving objects phase.
resolving)

Overrides org.eclipse.jgit.transport.PackParser.parse.

Parse the pack stream.

private int
read(long pos, byte[] dst, int off, int len)

protected int
readDatabase(byte[]
the buffer to copy read data into.
dst, int
position within dst to start copying data into.
pos, int
ideal target number of bytes to read. Actual read length may be shorter.
cnt)

Implements abstract org.eclipse.jgit.transport.PackParser.readDatabase.

Read from the database's current position into the buffer.

protected PackParser.ObjectTypeAndSize
seekDatabase(PackedObjectInfo
the object position to begin reading from. This is from newInfo(AnyObjectId, UnresolvedDelta, ObjectId).
obj, PackParser.ObjectTypeAndSize
object to populate with type and size.
info)

Implements abstract org.eclipse.jgit.transport.PackParser.seekDatabase.

Reposition the database to re-read a previously stored object.

protected PackParser.ObjectTypeAndSize
seekDatabase(PackParser.UnresolvedDelta
the object position to begin reading from. This is an instance previously returned by onEndDelta().
delta, PackParser.ObjectTypeAndSize
object to populate with type and size.
info)

Implements abstract org.eclipse.jgit.transport.PackParser.seekDatabase.

Reposition the database to re-read a previously stored object.

private long
toBlockStart(long pos)

private void
writePackIndex()

Inherited from org.eclipse.jgit.transport.PackParser:
buffergetBaseObjectIdsgetLockMessagegetNewObjectIdsgetObjectgetObjectCountgetPackSizegetReceivedPackStatisticsgetSortedObjectListisAllowThinisCheckEofAfterPackFooterisCheckObjectCollisionsisExpectDataAfterPackFooternewInfoparsereadObjectHeadersetAllowThinsetCheckEofAfterPackFootersetCheckObjectCollisionssetExpectDataAfterPackFootersetExpectedObjectCountsetLockMessagesetMaxObjectSizeLimitsetNeedBaseObjectIdssetNeedNewObjectIdssetObjectCheckersetObjectCheckingverifySafeObject

Field Detail

blockCacheback to summary
private DfsBlockCache blockCache

Cache the chunks were stored into or get read back from.

blockSizeback to summary
private int blockSize

Block size to use when caching data for read back.

crcback to summary
private final CRC32 crc

CRC-32 computation for objects that are appended onto the pack.

currBufback to summary
private byte[] currBuf

Data being written that has not yet been cached.

currEndback to summary
private int currEnd
currPosback to summary
private long currPos
defback to summary
private Deflater def

Compresses delta bases when completing a thin pack.

isEmptyPackback to summary
private boolean isEmptyPack

True if the pack is an empty pack.

objdbback to summary
private final DfsObjDatabase objdb
objinsback to summary
private final DfsInserter objins
outback to summary
private DfsOutputStream out

Stream to the DFS storage, opened during onPackHeader(long).

packDigestback to summary
private final MessageDigest packDigest

Hides org.eclipse.jgit.transport.PackParser.packDigest.

Running SHA-1 of the entire pack stream.

packDscback to summary
private DfsPackDescription packDsc

Name of the pack file, computed in onPackHeader(long).

packEndback to summary
private long packEnd

Current end of the pack file.

packHashback to summary
private byte[] packHash

Checksum of the entire pack file.

packIndexback to summary
private PackIndex packIndex

If the index was small enough, the entire index after writing.

packKeyback to summary
private DfsStreamKey packKey

Key used during delta resolution reading delta chains.

readBlockback to summary
private DfsBlock readBlock
readPosback to summary
private long readPos

Cached block that is being read.

Constructor Detail

DfsPackParserback to summary
protected DfsPackParser(DfsObjDatabase db, DfsInserter ins, InputStream in)

Initialize a new pack parser.

Parameters
db:DfsObjDatabase

database the objects will be imported into.

ins:DfsInserter

inserter the parser will use to help it inject the objects.

in:InputStream

the stream to parse.

Method Detail

bufferback to summary
private void buffer(byte[] raw, int pos, int len) throws IOException
checkCRCback to summary
protected boolean checkCRC(int oldCRC)

Implements abstract org.eclipse.jgit.transport.PackParser.checkCRC.

Doc from org.eclipse.jgit.transport.PackParser.checkCRC.

Check the current CRC matches the expected value.

This method is invoked when an object is read back in from the database and its data is used during delta resolution. The CRC is validated after the object has been fully read, allowing the parser to verify there was no silent data corruption.

Implementations are free to ignore this check by always returning true if they are performing other data integrity validations at a lower level.

Parameters
oldCRC:int

the prior CRC that was recorded during the first scan of the object from the pack stream.

Returns:boolean

true if the CRC matches; false if it does not.

Annotations
@Override
flushBlockback to summary
private DfsBlock flushBlock() throws IOException
getPackDescriptionback to summary
public DfsPackDescription getPackDescription()

Get description of the imported pack, if one was made.

Returns:DfsPackDescription

description of the imported pack, if one was made.

onAppendBaseback to summary
protected boolean onAppendBase(final int typeCode, final byte[] data, final PackedObjectInfo info) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onAppendBase.

Doc from org.eclipse.jgit.transport.PackParser.onAppendBase.

Provide the implementation with a base that was outside of the pack.

This event only occurs on a thin pack for base objects that were outside of the pack and came from the local repository. Usually an implementation uses this event to compress the base and append it onto the end of the pack, so the pack stays self-contained.

Parameters
typeCode:int

type of the base object.

data:byte[]

complete content of the base object.

info:PackedObjectInfo

packed object information for this base. Implementors must populate the CRC and offset members if returning true.

Returns:boolean

true if the info should be included in the object list returned by getSortedObjectList(Comparator), false if it should not be included.

Annotations
@Override
Exceptions
IOException:
the base could not be included into the pack.
onBeginOfsDeltaback to summary
protected void onBeginOfsDelta(long streamPosition, long baseStreamPosition, long inflatedSize) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginOfsDelta.

Doc from org.eclipse.jgit.transport.PackParser.onBeginOfsDelta.

Event notifying start of a delta referencing its base by offset.

Parameters
streamPosition:long

position of this object in the incoming stream.

baseStreamPosition:long

position of the base object in the incoming stream. The base must be before the delta, therefore baseStreamPosition < deltaStreamPosition. This is not the position returned by a prior end object event.

inflatedSize:long

size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onBeginRefDeltaback to summary
protected void onBeginRefDelta(long streamPosition, AnyObjectId baseId, long inflatedSize) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginRefDelta.

Doc from org.eclipse.jgit.transport.PackParser.onBeginRefDelta.

Event notifying start of a delta referencing its base by ObjectId.

Parameters
streamPosition:long

position of this object in the incoming stream.

baseId:AnyObjectId

name of the base object. This object may be later in the stream, or might not appear at all in the stream (in the case of a thin-pack).

inflatedSize:long

size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onBeginWholeObjectback to summary
protected void onBeginWholeObject(long streamPosition, int type, long inflatedSize) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onBeginWholeObject.

Doc from org.eclipse.jgit.transport.PackParser.onBeginWholeObject.

Event notifying the start of an object stored whole (not as a delta).

Parameters
streamPosition:long

position of this object in the incoming stream.

type:int

type of the object; one of org.eclipse.jgit.lib.Constants#OBJ_COMMIT, org.eclipse.jgit.lib.Constants#OBJ_TREE, org.eclipse.jgit.lib.Constants#OBJ_BLOB, or org.eclipse.jgit.lib.Constants#OBJ_TAG.

inflatedSize:long

size of the object when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onEndDeltaback to summary
protected PackParser.UnresolvedDelta onEndDelta() throws IOException

Overrides org.eclipse.jgit.transport.PackParser.onEndDelta.

Doc from org.eclipse.jgit.transport.PackParser.onEndDelta.

Event notifying the current object.

Returns:PackParser.UnresolvedDelta

object information that must be populated with at least the offset.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onEndThinPackback to summary
protected void onEndThinPack() throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onEndThinPack.

Doc from org.eclipse.jgit.transport.PackParser.onEndThinPack.

Event indicating a thin pack has been completely processed.

This event is invoked only if a thin pack has delta references to objects external from the pack. The event is called after all of those deltas have been resolved.

Annotations
@Override
Exceptions
IOException:
the pack cannot be archived.
onEndWholeObjectback to summary
protected void onEndWholeObject(PackedObjectInfo info) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onEndWholeObject.

Doc from org.eclipse.jgit.transport.PackParser.onEndWholeObject.

Event notifying the current object.

Parameters
info:PackedObjectInfo

object information.

Annotations
@Override
Exceptions
IOException:
the object cannot be recorded.
onInflatedObjectDataback to summary
protected void onInflatedObjectData(PackedObjectInfo obj, int typeCode, byte[] data) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onInflatedObjectData.

Doc from org.eclipse.jgit.transport.PackParser.onInflatedObjectData.

Invoked for commits, trees, tags, and small blobs.

Parameters
obj:PackedObjectInfo

the object info, populated.

typeCode:int

the type of the object.

data:byte[]

inflated data for the object.

Annotations
@Override
Exceptions
IOException:
the object cannot be archived.
onObjectDataback to summary
protected void onObjectData(PackParser.Source src, byte[] raw, int pos, int len) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onObjectData.

Doc from org.eclipse.jgit.transport.PackParser.onObjectData.

Store (and/or checksum) a portion of an object's data.

This method may be invoked multiple times per object, depending on the size of the object, the size of the parser's internal read buffer, and the alignment of the object relative to the read buffer.

Invoked after onObjectHeader(Source, byte[], int, int).

Parameters
src:PackParser.Source

where the data came from

raw:byte[]

buffer to read data from.

pos:int

first offset within buffer that is valid.

len:int

number of bytes in buffer that are valid.

Annotations
@Override
Exceptions
IOException:
the stream cannot be archived.
onObjectHeaderback to summary
protected void onObjectHeader(PackParser.Source src, byte[] raw, int pos, int len) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onObjectHeader.

Doc from org.eclipse.jgit.transport.PackParser.onObjectHeader.

Store (and/or checksum) an object header.

Invoked after any of the onBegin() events. The entire header is supplied in a single invocation, before any object data is supplied.

Parameters
src:PackParser.Source

where the data came from

raw:byte[]

buffer to read data from.

pos:int

first offset within buffer that is valid.

len:int

number of bytes in buffer that are valid.

Annotations
@Override
Exceptions
IOException:
the stream cannot be archived.
onPackFooterback to summary
protected void onPackFooter(byte[] hash) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onPackFooter.

Doc from org.eclipse.jgit.transport.PackParser.onPackFooter.

Provide the implementation with the original stream's pack footer.

Parameters
hash:byte[]

the trailing 20 bytes of the pack, this is a SHA-1 checksum of all of the pack data.

Annotations
@Override
Exceptions
IOException:
the stream cannot be archived.
onPackHeaderback to summary
protected void onPackHeader(long objectCount) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onPackHeader.

Doc from org.eclipse.jgit.transport.PackParser.onPackHeader.

Provide the implementation with the original stream's pack header.

Parameters
objectCount:long

number of objects expected in the stream.

Annotations
@Override
Exceptions
IOException:
the implementation refuses to work with this many objects.
onStoreStreamback to summary
protected void onStoreStream(byte[] raw, int pos, int len) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.onStoreStream.

Doc from org.eclipse.jgit.transport.PackParser.onStoreStream.

Store bytes received from the raw stream.

This method is invoked during parse(ProgressMonitor) as data is consumed from the incoming stream. Implementors may use this event to archive the raw incoming stream to the destination repository in large chunks, without paying attention to object boundaries.

The only component of the pack not supplied to this method is the last 20 bytes of the pack that comprise the trailing SHA-1 checksum. Those are passed to onPackFooter(byte[]).

Parameters
raw:byte[]

buffer to copy data out of.

pos:int

first offset within the buffer that is valid.

len:int

number of bytes in the buffer that are valid.

Annotations
@Override
Exceptions
IOException:
the stream cannot be archived.
parseback to summary
public PackLock parse(ProgressMonitor receiving, ProgressMonitor resolving) throws IOException

Overrides org.eclipse.jgit.transport.PackParser.parse.

Doc from org.eclipse.jgit.transport.PackParser.parse.

Parse the pack stream.

Parameters
receiving:ProgressMonitor

receives progress feedback during the initial receiving objects phase. If null, org.eclipse.jgit.lib.NullProgressMonitor will be used.

resolving:ProgressMonitor

receives progress feedback during the resolving objects phase.

Returns:PackLock

the pack lock, if one was requested by setting setLockMessage(String).

Annotations
@Override
Exceptions
IOException:
the stream is malformed, or contains corrupt objects.
readback to summary
private int read(long pos, byte[] dst, int off, int len) throws IOException
readDatabaseback to summary
protected int readDatabase(byte[] dst, int pos, int cnt) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.readDatabase.

Doc from org.eclipse.jgit.transport.PackParser.readDatabase.

Read from the database's current position into the buffer.

Parameters
dst:byte[]

the buffer to copy read data into.

pos:int

position within dst to start copying data into.

cnt:int

ideal target number of bytes to read. Actual read length may be shorter.

Returns:int

number of bytes stored.

Annotations
@Override
Exceptions
IOException:
the database cannot be accessed.
seekDatabaseback to summary
protected PackParser.ObjectTypeAndSize seekDatabase(PackedObjectInfo obj, PackParser.ObjectTypeAndSize info) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.seekDatabase.

Doc from org.eclipse.jgit.transport.PackParser.seekDatabase.

Reposition the database to re-read a previously stored object.

If the database is computing CRC-32 checksums for object data, it should reset its internal CRC instance during this method call.

Parameters
obj:PackedObjectInfo

the object position to begin reading from. This is from newInfo(AnyObjectId, UnresolvedDelta, ObjectId).

info:PackParser.ObjectTypeAndSize

object to populate with type and size.

Returns:PackParser.ObjectTypeAndSize

the info object.

Annotations
@Override
Exceptions
IOException:
the database cannot reposition to this location.
seekDatabaseback to summary
protected PackParser.ObjectTypeAndSize seekDatabase(PackParser.UnresolvedDelta delta, PackParser.ObjectTypeAndSize info) throws IOException

Implements abstract org.eclipse.jgit.transport.PackParser.seekDatabase.

Doc from org.eclipse.jgit.transport.PackParser.seekDatabase.

Reposition the database to re-read a previously stored object.

If the database is computing CRC-32 checksums for object data, it should reset its internal CRC instance during this method call.

Parameters
delta:PackParser.UnresolvedDelta

the object position to begin reading from. This is an instance previously returned by onEndDelta().

info:PackParser.ObjectTypeAndSize

object to populate with type and size.

Returns:PackParser.ObjectTypeAndSize

the info object.

Annotations
@Override
Exceptions
IOException:
the database cannot reposition to this location.
toBlockStartback to summary
private long toBlockStart(long pos)
writePackIndexback to summary
private void writePackIndex() throws IOException