Top Description Inners Fields Constructors Methods
org.eclipse.jgit.internal.storage.dfs

pack-priv final Class ClockBlockCacheTable

extends Object
implements DfsBlockCacheTable
Class Inheritance
All Implemented Interfaces
org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable
Imports
java.io.IOException, java.time.Duration, java.util.List, .Map, java.util.concurrent.ConcurrentHashMap, java.util.concurrent.atomic.AtomicReferenceArray, java.util.concurrent.locks.ReentrantLock, java.util.function.Consumer, java.util.stream.LongStream, org.eclipse.jgit.internal.JGitText, org.eclipse.jgit.internal.storage.dfs.DfsBlockCache.ReadableChannelSupplier, .DfsBlockCache.Ref, .DfsBlockCache.RefLoader, org.eclipse.jgit.internal.storage.pack.PackExt
Default implementation of the DfsBlockCacheTable.

This cache implements a clock replacement algorithm, giving each block at least one chance to have been accessed during a sweep of the cache to save itself from eviction. The number of swipe chances is configurable per pack extension.

Entities created by the cache are held under hard references, preventing the Java VM from clearing anything. Blocks are discarded by the replacement algorithm when adding a new block would cause the cache to exceed its configured maximum size.

Whenever a cache miss occurs, loading is invoked by exactly one thread for the given (DfsStreamKey,position) key tuple. This is ensured by an array of locks, with the tuple hashed to a lock instance.

The internal hash table does not expand at runtime, instead it is fixed in size at cache creation time. The internal lock table used to gate load invocations is also fixed in size.

Nested and Inner Type Summary

Modifier and TypeClass and Description
private static class
ClockBlockCacheTable.EvictKey

private static class
ClockBlockCacheTable.Hash

private static class
ClockBlockCacheTable.HashEntry

Field Summary

Modifier and TypeField and Description
private final int
blockSize

Used to reserve space for blocks.

private DfsBlockCache.Ref<T>
clockHand

Current position of the clock.

private final ReentrantLock
clockLock

Protects the clock and its related data.

private final DfsBlockCacheStats
dfsBlockCacheStats

private final ClockBlockCacheTable.Hash
hash

private final DfsBlockCacheConfig.IndexEventConsumer
indexEventConsumer

Consumer of loading and eviction events of indexes.

private final Map<ClockBlockCacheTable.EvictKey, Long>
indexEvictionMap

Stores timestamps of the last eviction of indexes.

private final ReentrantLock[]
loadLocks

Locks to prevent concurrent loads for same (PackFile,position) block.

private final long
maxBytes

Maximum number of bytes the cache should hold.

private final String
name

Table name.

private final ReentrantLock[][]
refLocks

A separate pool of locks per pack extension to prevent concurrent loads for same index or bitmap from PackFile.

private final Consumer<Long>
refLockWaitTime

A consumer of object reference lock wait time milliseconds.

private final AtomicReferenceArray<ClockBlockCacheTable.HashEntry>
table

Hash bucket directory; entries are chained below.

private final int
tableSize

Number of entries in table.

Constructor Summary

AccessConstructor and Description
pack-priv
ClockBlockCacheTable(DfsBlockCacheConfig cfg)

Method Summary

Modifier and TypeMethod and Description
private void
addToClock(DfsBlockCache.Ref<T> ref, long credit)

public boolean
contains(DfsStreamKey
the stream key of the pack.
key, long
the position in the key.
position)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.contains.

Check if the block cache contains an object identified by (key, position).

private ClockBlockCacheTable.EvictKey
createEvictKey(DfsBlockCache.Ref<?> ref)

private void
creditSpace(long credit, DfsStreamKey key)

public <T> T
get(DfsStreamKey
the stream key of the pack.
key, long
the position in the key.
position)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.get.

Get the object identified by (key, position) from the block cache.

public List<DfsBlockCacheTable.BlockCacheStats>
getBlockCacheStats()

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getBlockCacheStats.

Get the list of BlockCacheStats held by this cache.

public String
getName()

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getName.

Get the name of the table.

public DfsBlock
getOrLoad(BlockBasedFile
the pack that "contains" the cached object.
file, long
offset within pack of the object.
position, DfsReader
current thread's reader.
ctx, DfsBlockCache.ReadableChannelSupplier
supplier for channel to read pack.
fileChannel)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getOrLoad.

Look up a cached object, creating and loading it if it doesn't exist.

public <T> DfsBlockCache.Ref<T>
getOrLoadRef(DfsStreamKey
the stream key of the pack.
key, long
the position in the key. The default should be 0.
position, DfsBlockCache.RefLoader<T>
the function to load the reference.
loader)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getOrLoadRef.

Look up a cached object, creating and loading it if it doesn't exist.

public boolean
hasBlock0(DfsStreamKey
the file to check.
key)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.hasBlock0.

Quickly check if the cache contains block 0 of the given stream.

private static boolean
isIndexExtPos(int packExtPos)

private ReentrantLock
lockFor(DfsStreamKey key, long position)

private ReentrantLock
lockForRef(DfsStreamKey key)

public void
put(DfsBlock
the block to put in the cache.
v)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.put.

Put a block in the block cache.

public <T> DfsBlockCache.Ref<T>
put(DfsStreamKey
the stream key of the pack.
key, long
the position in the key.
pos, long
the size of the object.
size, T
the object to put in the block cache.
v)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.put.

Put a block in the block cache.

public <T> DfsBlockCache.Ref<T>
putRef(DfsStreamKey
the stream key of the pack.
key, long
the size of the object.
size, T
the object to put in the block cache.
v)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.putRef.

Put an object in the block cache.

private void
reportIndexEvicted(DfsBlockCache.Ref<?> dead)

private void
reportIndexRequested(DfsBlockCache.Ref<?> ref, boolean cacheHit, long start)

private void
reserveSpace(long reserve, DfsStreamKey key)

private <T> T
scan(ClockBlockCacheTable.HashEntry n, DfsStreamKey key, long position)

private <T> DfsBlockCache.Ref<T>
scanRef(ClockBlockCacheTable.HashEntry n, DfsStreamKey key, long position)

private int
slot(DfsStreamKey key, long position)

private static int
tableSize(DfsBlockCacheConfig cfg)

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

blockSizeback to summary
private final int blockSize

Used to reserve space for blocks.

The value for blockSize must be a power of 2.

clockHandback to summary
private DfsBlockCache.Ref<T> clockHand

Current position of the clock.

clockLockback to summary
private final ReentrantLock clockLock

Protects the clock and its related data.

dfsBlockCacheStatsback to summary
private final DfsBlockCacheStats dfsBlockCacheStats
hashback to summary
private final ClockBlockCacheTable.Hash hash
indexEventConsumerback to summary
private final DfsBlockCacheConfig.IndexEventConsumer indexEventConsumer

Consumer of loading and eviction events of indexes.

indexEvictionMapback to summary
private final Map<ClockBlockCacheTable.EvictKey, Long> indexEvictionMap

Stores timestamps of the last eviction of indexes.

loadLocksback to summary
private final ReentrantLock[] loadLocks

Locks to prevent concurrent loads for same (PackFile,position) block. The number of locks is DfsBlockCacheConfig#getConcurrencyLevel() to cap the overall concurrent block loads.

maxBytesback to summary
private final long maxBytes

Maximum number of bytes the cache should hold.

nameback to summary
private final String name

Table name.

refLocksback to summary
private final ReentrantLock[][] refLocks

A separate pool of locks per pack extension to prevent concurrent loads for same index or bitmap from PackFile.

refLockWaitTimeback to summary
private final Consumer<Long> refLockWaitTime

A consumer of object reference lock wait time milliseconds. May be used to build a metric.

tableback to summary
private final AtomicReferenceArray<ClockBlockCacheTable.HashEntry> table

Hash bucket directory; entries are chained below.

tableSizeback to summary
private final int tableSize

Number of entries in table.

Constructor Detail

ClockBlockCacheTableback to summary
pack-priv ClockBlockCacheTable(DfsBlockCacheConfig cfg)

Method Detail

addToClockback to summary
private void addToClock(DfsBlockCache.Ref<T> ref, long credit)
Annotations
@SuppressWarnings:unchecked
containsback to summary
public boolean contains(DfsStreamKey key, long position)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.contains.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.contains.

Check if the block cache contains an object identified by (key, position).

Parameters
key:DfsStreamKey

the stream key of the pack.

position:long

the position in the key.

Returns:boolean

if the block cache contains the object identified by (key, position).

Annotations
@Override
createEvictKeyback to summary
private ClockBlockCacheTable.EvictKey createEvictKey(DfsBlockCache.Ref<?> ref)
creditSpaceback to summary
private void creditSpace(long credit, DfsStreamKey key)
getback to summary
public <T> T get(DfsStreamKey key, long position)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.get.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.get.

Get the object identified by (key, position) from the block cache.

Parameters
key:DfsStreamKey

the stream key of the pack.

position:long

the position in the key.

Returns:T

the object identified by (key, position).

Annotations
@SuppressWarnings:unchecked
@Override
getBlockCacheStatsback to summary
public List<DfsBlockCacheTable.BlockCacheStats> getBlockCacheStats()

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getBlockCacheStats.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getBlockCacheStats.

Get the list of BlockCacheStats held by this cache.

The returned list has a BlockCacheStats per configured cache table, with a minimum of 1 BlockCacheStats object returned. Use AggregatedBlockCacheStats to combine the results of the stats in the list for an aggregated view of the cache's stats.

Returns:List<DfsBlockCacheTable.BlockCacheStats>

the list of BlockCacheStats held by this cache.

Annotations
@Override
getNameback to summary
public String getName()

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getName.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getName.

Get the name of the table.

Returns:String

this table's name.

Annotations
@Override
getOrLoadback to summary
public DfsBlock getOrLoad(BlockBasedFile file, long position, DfsReader ctx, DfsBlockCache.ReadableChannelSupplier fileChannel) throws IOException

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getOrLoad.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getOrLoad.

Look up a cached object, creating and loading it if it doesn't exist.

Parameters
file:BlockBasedFile

the pack that "contains" the cached object.

position:long

offset within pack of the object.

ctx:DfsReader

current thread's reader.

fileChannel:DfsBlockCache.ReadableChannelSupplier

supplier for channel to read pack.

Returns:DfsBlock

the object reference.

Annotations
@Override
Exceptions
IOException:
the reference was not in the cache and could not be loaded.
getOrLoadRefback to summary
public <T> DfsBlockCache.Ref<T> getOrLoadRef(DfsStreamKey key, long position, DfsBlockCache.RefLoader<T> loader) throws IOException

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getOrLoadRef.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.getOrLoadRef.

Look up a cached object, creating and loading it if it doesn't exist.

Parameters
key:DfsStreamKey

the stream key of the pack.

position:long

the position in the key. The default should be 0.

loader:DfsBlockCache.RefLoader<T>

the function to load the reference.

Returns:DfsBlockCache.Ref<T>

the object reference.

Annotations
@Override
Exceptions
IOException:
the reference was not in the cache and could not be loaded.
hasBlock0back to summary
public boolean hasBlock0(DfsStreamKey key)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.hasBlock0.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.hasBlock0.

Quickly check if the cache contains block 0 of the given stream.

This can be useful for sophisticated pre-read algorithms to quickly determine if a file is likely already in cache, especially small reftables which may be smaller than a typical DFS block size.

Parameters
key:DfsStreamKey

the file to check.

Returns:boolean

true if block 0 (the first block) is in the cache.

Annotations
@Override
isIndexExtPosback to summary
private static boolean isIndexExtPos(int packExtPos)
lockForback to summary
private ReentrantLock lockFor(DfsStreamKey key, long position)
lockForRefback to summary
private ReentrantLock lockForRef(DfsStreamKey key)
putback to summary
public void put(DfsBlock v)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.put.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.put.

Put a block in the block cache.

Parameters
v:DfsBlock

the block to put in the cache.

Annotations
@Override
putback to summary
public <T> DfsBlockCache.Ref<T> put(DfsStreamKey key, long pos, long size, T v)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.put.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.put.

Put a block in the block cache.

Parameters
key:DfsStreamKey

the stream key of the pack.

pos:long

the position in the key.

size:long

the size of the object.

v:T

the object to put in the block cache.

Returns:DfsBlockCache.Ref<T>

the object reference.

Annotations
@Override
putRefback to summary
public <T> DfsBlockCache.Ref<T> putRef(DfsStreamKey key, long size, T v)

Implements org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.putRef.

Doc from org.eclipse.jgit.internal.storage.dfs.DfsBlockCacheTable.putRef.

Put an object in the block cache.

Parameters
key:DfsStreamKey

the stream key of the pack.

size:long

the size of the object.

v:T

the object to put in the block cache.

Returns:DfsBlockCache.Ref<T>

the object reference.

Annotations
@Override
reportIndexEvictedback to summary
private void reportIndexEvicted(DfsBlockCache.Ref<?> dead)
reportIndexRequestedback to summary
private void reportIndexRequested(DfsBlockCache.Ref<?> ref, boolean cacheHit, long start)
reserveSpaceback to summary
private void reserveSpace(long reserve, DfsStreamKey key)
Annotations
@SuppressWarnings:unchecked
scanback to summary
private <T> T scan(ClockBlockCacheTable.HashEntry n, DfsStreamKey key, long position)
scanRefback to summary
private <T> DfsBlockCache.Ref<T> scanRef(ClockBlockCacheTable.HashEntry n, DfsStreamKey key, long position)
Annotations
@SuppressWarnings:unchecked
slotback to summary
private int slot(DfsStreamKey key, long position)
tableSizeback to summary
private static int tableSize(DfsBlockCacheConfig cfg)
org.eclipse.jgit.internal.storage.dfs back to summary

private final Class ClockBlockCacheTable.EvictKey

extends Object
Class Inheritance
  • java.lang.Object
  • org.eclipse.jgit.internal.storage.dfs.ClockBlockCacheTable.EvictKey

Field Summary

Modifier and TypeField and Description
private final ClockBlockCacheTable.Hash
hash

Provides the hash function to be used for this key's hashCode method.

private final int
keyHash

private final int
packExtPos

private final long
position

Constructor Summary

AccessConstructor and Description
pack-priv
EvictKey(ClockBlockCacheTable.Hash hash, DfsBlockCache.Ref<?> ref)

Method Summary

Modifier and TypeMethod and Description
public boolean
equals(Object
the reference object with which to compare.
object)

Overrides java.lang.Object.equals.

Indicates whether some other object is "equal to" this one.

public int
hashCode()

Overrides java.lang.Object.hashCode.

Returns a hash code value for this object.

Inherited from java.lang.Object:
clonefinalizegetClassnotifynotifyAlltoStringwaitwaitwait

Field Detail

hashback to summary
private final ClockBlockCacheTable.Hash hash

Provides the hash function to be used for this key's hashCode method.

keyHashback to summary
private final int keyHash
packExtPosback to summary
private final int packExtPos
positionback to summary
private final long position

Constructor Detail

EvictKeyback to summary
pack-priv EvictKey(ClockBlockCacheTable.Hash hash, DfsBlockCache.Ref<?> ref)

Method Detail

equalsback to summary
public boolean equals(Object object)

Overrides java.lang.Object.equals.

Doc from java.lang.Object.equals.

Indicates whether some other object is "equal to" this one.

The equals method implements an equivalence relation on non-null object references:

  • It is reflexive: for any non-null reference value x, x.equals(x) should return true.
  • It is symmetric: for any non-null reference values x and y, x.equals(y) should return true if and only if y.equals(x) returns true.
  • It is transitive: for any non-null reference values x, y, and z, if x.equals(y) returns true and y.equals(z) returns true, then x.equals(z) should return true.
  • It is consistent: for any non-null reference values x and y, multiple invocations of x.equals(y) consistently return true or consistently return false, provided no information used in equals comparisons on the objects is modified.
  • For any non-null reference value x, x.equals(null) should return false.

An equivalence relation partitions the elements it operates on into equivalence classes; all the members of an equivalence class are equal to each other. Members of an equivalence class are substitutable for each other, at least for some purposes.

Parameters
object:Object

the reference object with which to compare.

Returns:boolean

true if this object is the same as the obj argument; false otherwise.

Annotations
@Override
hashCodeback to summary
public int hashCode()

Overrides java.lang.Object.hashCode.

Doc from java.lang.Object.hashCode.

Returns a hash code value for this object. This method is supported for the benefit of hash tables such as those provided by java.util.HashMap.

The general contract of hashCode is:

  • Whenever it is invoked on the same object more than once during an execution of a Java application, the hashCode method must consistently return the same integer, provided no information used in equals comparisons on the object is modified. This integer need not remain consistent from one execution of an application to another execution of the same application.
  • If two objects are equal according to the equals method, then calling the hashCode method on each of the two objects must produce the same integer result.
  • It is not required that if two objects are unequal according to the equals method, then calling the hashCode method on each of the two objects must produce distinct integer results. However, the programmer should be aware that producing distinct integer results for unequal objects may improve the performance of hash tables.
Returns:int

a hash code value for this object

Annotations
@Override
org.eclipse.jgit.internal.storage.dfs back to summary

private final Class ClockBlockCacheTable.Hash

extends Object
Class Inheritance
  • java.lang.Object
  • org.eclipse.jgit.internal.storage.dfs.ClockBlockCacheTable.Hash

Field Summary

Modifier and TypeField and Description
private final int
blockSizeShift

As blockSize is a power of 2, bits to shift for a / blockSize.

Constructor Summary

AccessConstructor and Description
pack-priv
Hash(int blockSizeShift)

Method Summary

Modifier and TypeMethod and Description
pack-priv int
hash(int packHash, long off)

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

blockSizeShiftback to summary
private final int blockSizeShift

As blockSize is a power of 2, bits to shift for a / blockSize.

Constructor Detail

Hashback to summary
pack-priv Hash(int blockSizeShift)

Method Detail

hashback to summary
pack-priv int hash(int packHash, long off)
org.eclipse.jgit.internal.storage.dfs back to summary

private final Class ClockBlockCacheTable.HashEntry

extends Object
Class Inheritance
  • java.lang.Object
  • org.eclipse.jgit.internal.storage.dfs.ClockBlockCacheTable.HashEntry

Field Summary

Modifier and TypeField and Description
pack-priv final ClockBlockCacheTable.HashEntry
next

Next entry in the hash table's chain list.

pack-priv final DfsBlockCache.Ref<T>
ref

The referenced object.

Constructor Summary

AccessConstructor and Description
pack-priv
HashEntry(ClockBlockCacheTable.HashEntry n, DfsBlockCache.Ref<T> r)

Method Summary

Modifier and TypeMethod and Description
private static ClockBlockCacheTable.HashEntry
clean(ClockBlockCacheTable.HashEntry top)

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

nextback to summary
pack-priv final ClockBlockCacheTable.HashEntry next

Next entry in the hash table's chain list.

refback to summary
pack-priv final DfsBlockCache.Ref<T> ref

The referenced object.

Constructor Detail

HashEntryback to summary
pack-priv HashEntry(ClockBlockCacheTable.HashEntry n, DfsBlockCache.Ref<T> r)

Method Detail

cleanback to summary
private static ClockBlockCacheTable.HashEntry clean(ClockBlockCacheTable.HashEntry top)