Top Description Inners Fields Constructors Methods
org.eclipse.jgit.internal.storage.midx

pack-priv Class PackIndexMerger

extends Object
Class Inheritance
Imports
java.util.ArrayList, .Iterator, .List, .Map, .NoSuchElementException, java.util.stream.Collectors, org.eclipse.jgit.internal.storage.file.PackIndex, org.eclipse.jgit.lib.AnyObjectId, .MutableObjectId

Collect the stats and offers an iterator over the union of n-pack indexes.

The multipack index is a list of (sha1, packid, offset) ordered by sha1. We can build it from the individual pack indexes (sha1, offset) ordered by sha1, with a simple merge ignoring duplicates.

This class encapsulates the merging logic and precalculates the stats that the index needs (like total count of objects). To limit memory consumption, it does the merge as it goes during the iteration and iterators use mutable entries. The stats of the combined index are calculated in an iteration at construction time.

Nested and Inner Type Summary

Modifier and TypeClass and Description
private static class
pack-priv static class
PackIndexMerger.MidxMutableEntry

Object returned by the iterator.

private static class
PackIndexMerger.MultiIndexIterator

Iterator over n-indexes in ObjectId order.

pack-priv static class
PackIndexMerger.PackIndexPeekIterator

Convenience around the PackIndex iterator to read the current value multiple times without consuming it.

Field Summary

Modifier and TypeField and Description
private final List<PackIndex>
private static final int
private static final long
private final boolean
private final int
private final List<String>
private final int

Constructor Summary

AccessConstructor and Description
pack-priv

Method Summary

Modifier and TypeMethod and Description
pack-priv Iterator<PackIndexMerger.MidxMutableEntry>

Returns:

an iterator in sha1 order without duplicates.
bySha1Iterator
()

Iterator over the merged indexes in sha1 order without duplicates

The returned entry in the iterator is mutable, callers should NOT keep a reference to it.

pack-priv int

Returns:

number of objects with offsets over 2^31-1
getOffsetsOver31BitsCount
()

How many object have offsets over 2^31-1

Per multipack index spec, IF there is large offset chunk, all this offsets should be there.

pack-priv int

Returns:

count of packs merged
getPackCount
()

How many packs are being merged

pack-priv List<String>

Returns:

List of pack names, in the order used by the merge.
getPackNames
()

List of pack names in alphabetical order.

pack-priv int

Returns:

object count of the merged index
getUniqueObjectCount
()

Object count of the merged index (i.e. without duplicates)

pack-priv boolean

Returns:

true if there is any object with offset > 2^32 -1
needsLargeOffsetsChunk
()

If any object in any of the indexes has an offset over 2^32-1

pack-priv Iterator<PackIndexMerger.MidxMutableEntry>

Returns:

an iterator of all objects in sha1 order, including duplicates.
rawIterator
()

For testing.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

indexesback to summary
private final List<PackIndex> indexes
LIMIT_31_BITSback to summary
private static final int LIMIT_31_BITS
LIMIT_32_BITSback to summary
private static final long LIMIT_32_BITS
needsLargeOffsetsChunkback to summary
private final boolean needsLargeOffsetsChunk
offsetsOver31BitsCountback to summary
private final int offsetsOver31BitsCount
packNamesback to summary
private final List<String> packNames
uniqueObjectCountback to summary
private final int uniqueObjectCount

Constructor Detail

PackIndexMergerback to summary
pack-priv PackIndexMerger(Map<String, PackIndex> packs)

Method Detail

bySha1Iteratorback to summary
pack-priv Iterator<PackIndexMerger.MidxMutableEntry> bySha1Iterator()

Iterator over the merged indexes in sha1 order without duplicates

The returned entry in the iterator is mutable, callers should NOT keep a reference to it.

Returns:Iterator<PackIndexMerger.MidxMutableEntry>

an iterator in sha1 order without duplicates.

getOffsetsOver31BitsCountback to summary
pack-priv int getOffsetsOver31BitsCount()

How many object have offsets over 2^31-1

Per multipack index spec, IF there is large offset chunk, all this offsets should be there.

Returns:int

number of objects with offsets over 2^31-1

getPackCountback to summary
pack-priv int getPackCount()

How many packs are being merged

Returns:int

count of packs merged

getPackNamesback to summary
pack-priv List<String> getPackNames()

List of pack names in alphabetical order.

Order matters: In case of duplicates, the multipack index prefers the first package with it. This is in the same order we are using to prioritize duplicates.

Returns:List<String>

List of pack names, in the order used by the merge.

getUniqueObjectCountback to summary
pack-priv int getUniqueObjectCount()

Object count of the merged index (i.e. without duplicates)

Returns:int

object count of the merged index

needsLargeOffsetsChunkback to summary
pack-priv boolean needsLargeOffsetsChunk()

If any object in any of the indexes has an offset over 2^32-1

Returns:boolean

true if there is any object with offset > 2^32 -1

rawIteratorback to summary
pack-priv Iterator<PackIndexMerger.MidxMutableEntry> rawIterator()

For testing. Iterate all entries, not skipping duplicates (stable order)

Returns:Iterator<PackIndexMerger.MidxMutableEntry>

an iterator of all objects in sha1 order, including duplicates.

org.eclipse.jgit.internal.storage.midx back to summary

private Class PackIndexMerger.DedupMultiIndexIterator

extends Object
implements Iterator<PackIndexMerger.MidxMutableEntry>
Class Inheritance
  • java.lang.Object
  • org.eclipse.jgit.internal.storage.midx.PackIndexMerger.DedupMultiIndexIterator
All Implemented Interfaces
java.util.Iterator

Field Summary

Modifier and TypeField and Description
private final MutableObjectId
private int
private final PackIndexMerger.MultiIndexIterator

Constructor Summary

AccessConstructor and Description
pack-priv

Method Summary

Modifier and TypeMethod and Description
public boolean
hasNext()

Implements java.util.Iterator.hasNext.

Returns true if the iteration has more elements.

public PackIndexMerger.MidxMutableEntry
next()

Implements java.util.Iterator.next.

Returns the next element in the iteration.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

lastOidback to summary
private final MutableObjectId lastOid
remainingback to summary
private int remaining
srcback to summary
private final PackIndexMerger.MultiIndexIterator src

Constructor Detail

DedupMultiIndexIteratorback to summary
pack-priv DedupMultiIndexIterator(PackIndexMerger.MultiIndexIterator src, int totalCount)

Method Detail

hasNextback to summary
public boolean hasNext()

Implements java.util.Iterator.hasNext.

Doc from java.util.Iterator.hasNext.

Returns true if the iteration has more elements. (In other words, returns true if next would return an element rather than throwing an exception.)

Returns:boolean

true if the iteration has more elements

Annotations
@Override
nextback to summary
public PackIndexMerger.MidxMutableEntry next()

Implements java.util.Iterator.next.

Doc from java.util.Iterator.next.

Returns the next element in the iteration.

Returns:PackIndexMerger.MidxMutableEntry

the next element in the iteration

Annotations
@Override
org.eclipse.jgit.internal.storage.midx back to summary

pack-priv Class PackIndexMerger.MidxMutableEntry

extends Object
Class Inheritance
  • java.lang.Object
  • org.eclipse.jgit.internal.storage.midx.PackIndexMerger.MidxMutableEntry

Object returned by the iterator.

The iterator returns (on each next()) the same instance with different values, to avoid allocating many short-lived objects. Callers should not keep a reference to that returned value.

Field Summary

Modifier and TypeField and Description
private long
private final MutableObjectId
private int

Constructor Summary

AccessConstructor and Description
pack-priv

Method Summary

Modifier and TypeMethod and Description
private void
fill(int
packId
packId
,
PackIndex.MutableEntry
another mutable entry
other
)

Copy values from another mutable entry

public AnyObjectId
public long
public int
Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

offsetback to summary
private long offset
oidback to summary
private final MutableObjectId oid
packIdback to summary
private int packId

Constructor Detail

MidxMutableEntryback to summary
pack-priv MidxMutableEntry()

Method Detail

fillback to summary
private void fill(int packId, PackIndex.MutableEntry other)

Copy values from another mutable entry

Parameters
packId:int

packId

other:PackIndex.MutableEntry

another mutable entry

getObjectIdback to summary
public AnyObjectId getObjectId()
getOffsetback to summary
public long getOffset()
getPackIdback to summary
public int getPackId()
org.eclipse.jgit.internal.storage.midx back to summary

private final Class PackIndexMerger.MultiIndexIterator

extends Object
implements Iterator<PackIndexMerger.MidxMutableEntry>
Class Inheritance
  • java.lang.Object
  • org.eclipse.jgit.internal.storage.midx.PackIndexMerger.MultiIndexIterator
All Implemented Interfaces
java.util.Iterator

Iterator over n-indexes in ObjectId order.

It returns duplicates if the same object id is in different indexes. Wrap it with (Iterator, int) to avoid duplicates.

Field Summary

Modifier and TypeField and Description
private final List<PackIndexMerger.PackIndexPeekIterator>
private final PackIndexMerger.MidxMutableEntry

Constructor Summary

AccessConstructor and Description
pack-priv

Method Summary

Modifier and TypeMethod and Description
public boolean
hasNext()

Implements java.util.Iterator.hasNext.

Returns true if the iteration has more elements.

public PackIndexMerger.MidxMutableEntry
next()

Implements java.util.Iterator.next.

Returns the next element in the iteration.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

indexIteratorsback to summary
private final List<PackIndexMerger.PackIndexPeekIterator> indexIterators
mutableEntryback to summary
private final PackIndexMerger.MidxMutableEntry mutableEntry

Constructor Detail

MultiIndexIteratorback to summary
pack-priv MultiIndexIterator(List<PackIndex> indexes)

Method Detail

hasNextback to summary
public boolean hasNext()

Implements java.util.Iterator.hasNext.

Doc from java.util.Iterator.hasNext.

Returns true if the iteration has more elements. (In other words, returns true if next would return an element rather than throwing an exception.)

Returns:boolean

true if the iteration has more elements

Annotations
@Override
nextback to summary
public PackIndexMerger.MidxMutableEntry next()

Implements java.util.Iterator.next.

Doc from java.util.Iterator.next.

Returns the next element in the iteration.

Returns:PackIndexMerger.MidxMutableEntry

the next element in the iteration

Annotations
@Override
org.eclipse.jgit.internal.storage.midx back to summary

pack-priv Class PackIndexMerger.PackIndexPeekIterator

extends Object
Class Inheritance
  • java.lang.Object
  • org.eclipse.jgit.internal.storage.midx.PackIndexMerger.PackIndexPeekIterator

Convenience around the PackIndex iterator to read the current value multiple times without consuming it.

This is used to merge indexes in the multipack index, where we need to compare the current value between indexes multiple times to find the next.

We could also implement this keeping the position (int) and MutableEntry#getObjectId, but that would create an ObjectId per entry. This implementation reuses the MutableEntry and avoid instantiations.

Field Summary

Modifier and TypeField and Description
pack-priv PackIndex.MutableEntry
private final Iterator<PackIndex.MutableEntry>
private final int

Constructor Summary

AccessConstructor and Description
pack-priv
PackIndexPeekIterator(int packId, PackIndex index)

Method Summary

Modifier and TypeMethod and Description
pack-priv int
pack-priv PackIndex.MutableEntry
next()

pack-priv PackIndex.MutableEntry
peek()

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

currentback to summary
pack-priv PackIndex.MutableEntry current
itback to summary
private final Iterator<PackIndex.MutableEntry> it
packIdback to summary
private final int packId

Constructor Detail

PackIndexPeekIteratorback to summary
pack-priv PackIndexPeekIterator(int packId, PackIndex index)

Method Detail

getPackIdback to summary
pack-priv int getPackId()
nextback to summary
pack-priv PackIndex.MutableEntry next()
peekback to summary
pack-priv PackIndex.MutableEntry peek()