A BackingStoreHashtable is a utility class which will store a set of rows into an in memory hash table, or overflow the hash table to a tempory on disk structure.

All rows must contain the same number of columns, and the column at position N of all the rows must have the same format id. If the BackingStoreHashtable needs to be overflowed to disk, then an arbitrary row will be chosen and used as a template for creating the underlying overflow container.

The hash table will be built logically as follows (actual implementation may differ). The important points are that the hash value is the standard java hash value on the row[key_column_numbers[0], if key_column_numbers.length is 1, or row[key_column_numbers[0, 1, ...]] if key_column_numbers.length > 1, and that duplicate detection is done by the standard java duplicate detection provided by java.util.Hashtable.

import java.util.Hashtable;

hash_table = new Hashtable();

Object row; // is a DataValueDescriptor[] or a LocatedRow

boolean  needsToClone = rowSource.needsToClone();

while((row = rowSource.getNextRowFromRowSource()) != null)
{
    if (needsToClone)
        row = clone_row_from_row(row);

    Object key = KeyHasher.buildHashKey(row, key_column_numbers);

    if ((duplicate_value = 
        hash_table.put(key, row)) != null)
    {
        Vector row_vec;

        // inserted a duplicate
        if ((duplicate_value instanceof vector))
        {
            row_vec = (Vector) duplicate_value;
        }
        else
        {
            // allocate vector to hold duplicates
            row_vec = new Vector(2);

            // insert original row into vector
            row_vec.addElement(duplicate_value);

            // put the vector as the data rather than the row
            hash_table.put(key, row_vec);
        }

        // insert new row into vector
        row_vec.addElement(row);
    }
}

What actually goes into the hash table is a little complicated. That is because the row may either be an array of column values (i.e. DataValueDescriptor[]) or a LocatedRow (i.e., a structure holding the columns plus a RowLocation). In addition, the hash value itself may either be one of these rows or (in the case of multiple rows which hash to the same value) a bucket (List) of rows. To sum this up, the values in a hash table which does not spill to disk may be the following:

DataValueDescriptor[] and ArrayList<DataValueDescriptor>
or LocatedRow and ArrayList<LocatedRow>

If rows spill to disk, then they just become arrays of columns. In this case, a LocatedRow becomes a DataValueDescriptor[], where the last cell contains the RowLocation.

Nested and Inner Type Summary

Modifier and Type	Class and Description
private class	BackingStoreHashtable.BackingStoreHashtableEnumeration
private static class	BackingStoreHashtable.RowList List of `DataValueDescriptor[]` instances that represent rows.

Field Summary

Modifier and Type	Field and Description
private static final int	ARRAY_LIST_SIZE The estimated number of bytes used by ArrayList(0)
private Properties	auxillary_runtimestats
private DiskHashtable	diskHashtable
private HashMap<Object, Object>	hash_table
private long	inmemory_rowcnt
private boolean	keepAfterCommit
private int[]	key_column_numbers
private long	max_inmemory_rowcnt
private long	max_inmemory_size
private boolean	remove_duplicates
private RowSource	row_source
private boolean	skipNullKeyColumns
private TransactionController	tc Fields of the class

Constructor Summary

Access	Constructor and Description
private	BackingStoreHashtable() Constructors for This class:
public	BackingStoreHashtable(TransactionController An open TransactionController to be used if the hash table needs to overflow to disk. tc, RowSource RowSource to read rows from. row_source, int[] The column numbers of the columns in the scan result row to be the key to the HashMap. "0" is the first column in the scan result row (which may be different than the first row in the table of the scan). key_column_numbers, boolean Should the HashMap automatically remove duplicates, or should it create the list of duplicates? remove_duplicates, long The estimated number of rows in the hash table. Pass in -1 if there is no estimate. estimated_rowcnt, long The maximum number of rows to insert into the inmemory Hash table before overflowing to disk. Pass in -1 if there is no maximum. max_inmemory_rowcnt, int If not "-1" used to initialize the java HashMap initialCapacity, float If not "-1" used to initialize the java HashMap loadFactor, boolean Skip rows with a null key column, if true. skipNullKeyColumns, boolean If true the hash table is kept after a commit, if false the hash table is dropped on the next commit. keepAfterCommit) Create the BackingStoreHashtable from a row source.

Access

Constructor and Description

private

BackingStoreHashtable()

Constructors for This class:

public

BackingStoreHashtable(TransactionController

An open TransactionController to be used if the hash table needs to overflow to disk.

tc, RowSource

RowSource to read rows from.

row_source, int[]

The column numbers of the columns in the scan result row to be the key to the HashMap. "0" is the first column in the scan result row (which may be different than the first row in the table of the scan).

key_column_numbers, boolean

Should the HashMap automatically remove duplicates, or should it create the list of duplicates?

remove_duplicates, long

The estimated number of rows in the hash table. Pass in -1 if there is no estimate.

estimated_rowcnt, long

The maximum number of rows to insert into the inmemory Hash table before overflowing to disk. Pass in -1 if there is no maximum.

max_inmemory_rowcnt, int

If not "-1" used to initialize the java HashMap

initialCapacity, float

If not "-1" used to initialize the java HashMap

loadFactor, boolean

Skip rows with a null key column, if true.

skipNullKeyColumns, boolean

If true the hash table is kept after a commit, if false the hash table is dropped on the next commit.

keepAfterCommit)

Create the BackingStoreHashtable from a row source.

Method Summary

Modifier and Type	Method and Description
private void	add_row_to_hash_table(DataValueDescriptor[] Row to add to the hash table. columnValues, RowLocation Location of row in conglomerate; could be null. rowLocation, boolean If the row needs to be cloned needsToClone) Do the work to add one row to the hash table.
private static DataValueDescriptor[]	Returns: The cloned row row to use. cloneRow(DataValueDescriptor[] old_row) Return a cloned copy of the row.
public void	close() Close the BackingStoreHashtable.
private void	doSpaceAccounting(Object hashValue, boolean firstDuplicate)
public Enumeration<Object>	Returns: The Enumeration. elements() Return an Enumeration that can be used to scan the entire table.
public Object	Returns: The value to which the key is mapped in this hashtable; null if the key is not mapped to any value in this hashtable. get(Object The key to hash on. key) Get data associated with given key.
public void	getAllRuntimeStats(Properties The set of properties to append to. prop) Return runtime stats to caller by adding them to prop.
private long	Returns: A guess as to how much memory the current hash value will use. getEstimatedMemUsage(Object The object for which we want to know the memory usage. hashValue) Take a value which will go into the hash table and return an estimate of how much memory that value will consume.
private DataValueDescriptor[]	getNextRowFromRowSource() Call method to either get next row or next row with non-null key columns.
public boolean	includeRowLocations() Return true if we should include RowLocations with the rows stored in this hash table.
private DataValueDescriptor[]	makeDiskRow(Object raw) Make a full set of columns from an object which is either already an array of column or otherwise a LocatedRow.
private DataValueDescriptor[]	makeDiskRow(DataValueDescriptor[] columnValues, RowLocation rowLocation) Construct a full set of columns, which may need to end with the row location.The full set of columns is what's stored on disk when we spill to disk.
private Object	makeInMemoryRow(DataValueDescriptor[] diskRow) Make an in-memory row from an on-disk row.
private List<E>	makeInMemoryRows(List<E> diskRows) Turn a list of disk rows into a list of in-memory rows.
public boolean	Returns: true if row was inserted into the hash table. Returns false if the BackingStoreHashtable is eliminating duplicates, and the row being inserted is a duplicate, or if we are skipping rows with 1 or more null key columns and we find a null key column. putRow(boolean does this routine have to make a copy of the row, in order to keep a reference to it after return? needsToClone, DataValueDescriptor[] The row to insert into the table. row, RowLocation Location of row in conglomerate; could be null. rowLocation) Put a row into the hash table.
public Object	remove(Object The key of the row to remove. key) remove a row from the hash table.
public void	setAuxillaryRuntimeStats(Properties The set of properties to append from. prop) Set the auxillary runtime stats.
pack-priv static DataValueDescriptor[]	Returns: The cloned row row to use. shallowCloneRow(DataValueDescriptor[] old_row) Return a shallow cloned row
public int	Returns: The number of unique rows in the hash table. size() Return number of unique rows in the hash table.
private boolean	Returns: true if the row was spilled to disk, false if not spillToDisk(DataValueDescriptor[] Actual columns from source row. columnValues, RowLocation Optional row location. rowLocation) Determine whether a new row should be spilled to disk and, if so, do it.

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

Field Detail

ARRAY_LIST_SIZE	back to summary
private static final int ARRAY_LIST_SIZE The estimated number of bytes used by ArrayList(0)

auxillary_runtimestats	back to summary
private Properties auxillary_runtimestats

diskHashtable	back to summary
private DiskHashtable diskHashtable

hash_table	back to summary
private HashMap<Object, Object> hash_table

inmemory_rowcnt	back to summary
private long inmemory_rowcnt

keepAfterCommit	back to summary
private boolean keepAfterCommit

key_column_numbers	back to summary
private int[] key_column_numbers

max_inmemory_rowcnt	back to summary
private long max_inmemory_rowcnt

max_inmemory_size	back to summary
private long max_inmemory_size

remove_duplicates	back to summary
private boolean remove_duplicates

row_source	back to summary
private RowSource row_source

skipNullKeyColumns	back to summary
private boolean skipNullKeyColumns

tc	back to summary
private TransactionController tc Fields of the class

Constructor Detail

BackingStoreHashtable	back to summary
private BackingStoreHashtable() Constructors for This class:

BackingStoreHashtable	back to summary
public BackingStoreHashtable(TransactionController tc, RowSource row_source, int[] key_column_numbers, boolean remove_duplicates, long estimated_rowcnt, long max_inmemory_rowcnt, int initialCapacity, float loadFactor, boolean skipNullKeyColumns, boolean keepAfterCommit) throws StandardException Create the BackingStoreHashtable from a row source. This routine drains the RowSource. The performance characteristics depends on the number of rows inserted and the parameters to the constructor. RowLocations are supported iff row_source is null. RowLocations in a non-null row_source can be added later if there is a use-case that stresses this behavior. If the number of rows is <= "max_inmemory_rowcnt", then the rows are inserted into a java.util.HashMap. In this case no TransactionController is necessary, a "null" tc is valid. If the number of rows is > "max_inmemory_rowcnt", then the rows will be all placed in some sort of Access temporary file on disk. This case requires a valid TransactionController. Parameters tc:TransactionController An open TransactionController to be used if the hash table needs to overflow to disk. row_source:RowSource RowSource to read rows from. key_column_numbers:int[] The column numbers of the columns in the scan result row to be the key to the HashMap. "0" is the first column in the scan result row (which may be different than the first row in the table of the scan). remove_duplicates:boolean Should the HashMap automatically remove duplicates, or should it create the list of duplicates? estimated_rowcnt:long The estimated number of rows in the hash table. Pass in -1 if there is no estimate. max_inmemory_rowcnt:long The maximum number of rows to insert into the inmemory Hash table before overflowing to disk. Pass in -1 if there is no maximum. initialCapacity:int If not "-1" used to initialize the java HashMap loadFactor:float If not "-1" used to initialize the java HashMap skipNullKeyColumns:boolean Skip rows with a null key column, if true. keepAfterCommit:boolean If true the hash table is kept after a commit, if false the hash table is dropped on the next commit. Exceptions StandardException: Standard exception policy.

BackingStoreHashtable

Modifier and Type	Field and Description
private Enumeration<Object>	diskEnumeration
private Iterator<Object>	memoryIterator

Modifier and Type	Method and Description
public boolean	hasMoreElements() Implements java.util.Enumeration.hasMoreElements. Tests if this enumeration contains more elements.
public Object	nextElement() Implements java.util.Enumeration.nextElement. Returns the next element of this enumeration if this enumeration object has at least one more element to provide.

public Class BackingStoreHashtable

Nested and Inner Type Summary

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail

private Class BackingStoreHashtable.BackingStoreHashtableEnumeration

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail

private Class BackingStoreHashtable.RowList

Constructor Summary

Method Summary

Constructor Detail