Top Description Inners Fields Constructors Methods
com.ibm.icu.impl.coll

pack-priv final Class CollationDataBuilder

extends Object
Class Inheritance
Imports
java.util.ArrayList, .Arrays, .Iterator, com.ibm.icu.impl.Norm2AllModes, .Normalizer2Impl, .Normalizer2Impl.Hangul, .Trie2, .Trie2Writable, com.ibm.icu.lang.UCharacter, com.ibm.icu.text.UnicodeSet, .UnicodeSetIterator, com.ibm.icu.util.CharsTrie, .CharsTrieBuilder, .StringTrieBuilder

Low-level CollationData builder. Takes (character, CE) pairs and builds them into runtime data structures. Supports characters with context prefixes and contraction suffixes.

Nested and Inner Type Summary

Modifier and TypeClass and Description
pack-priv static interface
CollationDataBuilder.CEModifier

Collation element modifier.

private static class
CollationDataBuilder.ConditionalCE32

Build-time context and CE32 for a code point.

private static class
private static class
CollationDataBuilder.DataBuilderCollationIterator

Build-time collation element and character iterator.

Field Summary

Modifier and TypeField and Description
protected CollationData
protected CollationSettings
protected UVector32
protected UVector64
protected CollationDataBuilder.DataBuilderCollationIterator
protected ArrayList<CollationDataBuilder.ConditionalCE32>
protected UnicodeSet
protected StringBuilder
private int
contextsEra

The "era" of building intermediate contexts.

protected CollationFastLatinBuilder
protected boolean
private static final int
protected boolean
protected Normalizer2Impl
protected Trie2Writable
protected UnicodeSet

Constructor Summary

AccessConstructor and Description
pack-priv

Method Summary

Modifier and TypeMethod and Description
pack-priv void
add(CharSequence prefix, CharSequence s, long[] ces, int cesLength)

protected int
addCE(long ce)

pack-priv void
addCE32(CharSequence prefix, CharSequence s, int ce32)

protected int
addCE32(int ce32)

protected int
addConditionalCE32(String context, int ce32)

protected int
addContextTrie(int defaultCE32, CharsTrieBuilder trieBuilder)

pack-priv void
protected int
protected void
protected void
protected void
protected void
protected int
copyContractionsFromBaseCE32(StringBuilder context, int c, int ce32, CollationDataBuilder.ConditionalCE32 cond)

Copies base contractions to a list of ConditionalCE32.

pack-priv void
copyFrom(CollationDataBuilder src, CollationDataBuilder.CEModifier modifier)

Copies all mappings from the src builder, with modifications.

protected int
copyFromBaseCE32(int c, int ce32, boolean withContext)

pack-priv void
pack-priv int
encodeCEs(long[] ces, int cesLength)

Encodes the ces as either the returned ce32 by itself, or by storing an expansion, with the returned ce32 referring to that.

protected int
encodeExpansion(long[] ces, int start, int length)

protected int
encodeExpansion32(int[] newCE32s, int start, int length)

protected int
encodeOneCE(long ce)

protected static int
private static void
enumRangeForCopy(int start, int end, int value, CollationDataBuilder.CopyHelper helper)

protected int
getCE32FromOffsetCE32(boolean fromBase, int c, int ce32)

pack-priv int

Returns:

incremented cesLength
getCEs
(CharSequence s, long[] ces, int cesLength)

Looks up CEs for s and appends them to the ces array.

pack-priv int
getCEs(CharSequence prefix, CharSequence s, long[] ces, int cesLength)

protected int
getCEs(CharSequence s, int start, long[] ces, int cesLength)

protected CollationDataBuilder.ConditionalCE32
getConditionalCE32(int index)

protected CollationDataBuilder.ConditionalCE32
protected boolean
getJamoCE32s(int[] jamoCE32s)

pack-priv boolean

Returns:

true if this builder has mappings (e.g., add() has been called)
hasMappings
()

pack-priv void
pack-priv boolean

Returns:

true if c has CEs in this builder
isAssigned
(int c)

protected static boolean
pack-priv boolean
pack-priv boolean
protected final boolean
protected static int
protected static int
pack-priv void
protected void
protected void
pack-priv void
Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

baseback to summary
protected CollationData base
baseSettingsback to summary
protected CollationSettings baseSettings
ce32sback to summary
protected UVector32 ce32s
ce64sback to summary
protected UVector64 ce64s
collIterback to summary
protected CollationDataBuilder.DataBuilderCollationIterator collIter
conditionalCE32sback to summary
protected ArrayList<CollationDataBuilder.ConditionalCE32> conditionalCE32s
contextCharsback to summary
protected UnicodeSet contextChars
contextsback to summary
protected StringBuilder contexts
contextsEraback to summary
private int contextsEra

The "era" of building intermediate contexts. When the array of cached, temporary contexts overflows, then clearContexts() removes them all and invalidates the builtCE32 that used to point to built tries. See ConditionalCE32#era.

fastLatinBuilderback to summary
protected CollationFastLatinBuilder fastLatinBuilder
fastLatinEnabledback to summary
protected boolean fastLatinEnabled
IS_BUILDER_JAMO_CE32back to summary
private static final int IS_BUILDER_JAMO_CE32
See Also
Collation.BUILDER_DATA_TAG
modifiedback to summary
protected boolean modified
nfcImplback to summary
protected Normalizer2Impl nfcImpl
trieback to summary
protected Trie2Writable trie
unsafeBackwardSetback to summary
protected UnicodeSet unsafeBackwardSet

Constructor Detail

CollationDataBuilderback to summary
pack-priv CollationDataBuilder()

Method Detail

addback to summary
pack-priv void add(CharSequence prefix, CharSequence s, long[] ces, int cesLength)
addCEback to summary
protected int addCE(long ce)
addCE32back to summary
pack-priv void addCE32(CharSequence prefix, CharSequence s, int ce32)
addCE32back to summary
protected int addCE32(int ce32)
addConditionalCE32back to summary
protected int addConditionalCE32(String context, int ce32)
addContextTrieback to summary
protected int addContextTrie(int defaultCE32, CharsTrieBuilder trieBuilder)
buildback to summary
pack-priv void build(CollationData data)
buildContextback to summary
protected int buildContext(CollationDataBuilder.ConditionalCE32 head)
buildContextsback to summary
protected void buildContexts()
buildFastLatinTableback to summary
protected void buildFastLatinTable(CollationData data)
buildMappingsback to summary
protected void buildMappings(CollationData data)
clearContextsback to summary
protected void clearContexts()
copyContractionsFromBaseCE32back to summary
protected int copyContractionsFromBaseCE32(StringBuilder context, int c, int ce32, CollationDataBuilder.ConditionalCE32 cond)

Copies base contractions to a list of ConditionalCE32. Sets cond.next to the index of the first new item and returns the index of the last new item.

copyFromback to summary
pack-priv void copyFrom(CollationDataBuilder src, CollationDataBuilder.CEModifier modifier)

Copies all mappings from the src builder, with modifications. This builder here must not be built yet, and should be empty.

copyFromBaseCE32back to summary
protected int copyFromBaseCE32(int c, int ce32, boolean withContext)
enableFastLatinback to summary
pack-priv void enableFastLatin()
encodeCEsback to summary
pack-priv int encodeCEs(long[] ces, int cesLength)

Encodes the ces as either the returned ce32 by itself, or by storing an expansion, with the returned ce32 referring to that.

add(p, s, ces, cesLength) = addCE32(p, s, encodeCEs(ces, cesLength))

encodeExpansionback to summary
protected int encodeExpansion(long[] ces, int start, int length)
encodeExpansion32back to summary
protected int encodeExpansion32(int[] newCE32s, int start, int length)
encodeOneCEback to summary
protected int encodeOneCE(long ce)
encodeOneCEAsCE32back to summary
protected static int encodeOneCEAsCE32(long ce)
enumRangeForCopyback to summary
private static void enumRangeForCopy(int start, int end, int value, CollationDataBuilder.CopyHelper helper)
getCE32FromOffsetCE32back to summary
protected int getCE32FromOffsetCE32(boolean fromBase, int c, int ce32)
getCEsback to summary
pack-priv int getCEs(CharSequence s, long[] ces, int cesLength)

Looks up CEs for s and appends them to the ces array. Does not handle normalization: s should be in FCD form. Does not write completely ignorable CEs. Does not write beyond Collation.MAX_EXPANSION_LENGTH.

Returns:int

incremented cesLength

getCEsback to summary
pack-priv int getCEs(CharSequence prefix, CharSequence s, long[] ces, int cesLength)
getCEsback to summary
protected int getCEs(CharSequence s, int start, long[] ces, int cesLength)
getConditionalCE32back to summary
protected CollationDataBuilder.ConditionalCE32 getConditionalCE32(int index)
getConditionalCE32ForCE32back to summary
protected CollationDataBuilder.ConditionalCE32 getConditionalCE32ForCE32(int ce32)
getJamoCE32sback to summary
protected boolean getJamoCE32s(int[] jamoCE32s)
hasMappingsback to summary
pack-priv boolean hasMappings()
Returns:boolean

true if this builder has mappings (e.g., add() has been called)

initForTailoringback to summary
pack-priv void initForTailoring(CollationData b)
isAssignedback to summary
pack-priv boolean isAssigned(int c)
Returns:boolean

true if c has CEs in this builder

isBuilderContextCE32back to summary
protected static boolean isBuilderContextCE32(int ce32)
isCompressibleLeadByteback to summary
pack-priv boolean isCompressibleLeadByte(int b)
isCompressiblePrimaryback to summary
pack-priv boolean isCompressiblePrimary(long p)
isMutableback to summary
protected final boolean isMutable()
jamoCpFromIndexback to summary
protected static int jamoCpFromIndex(int i)
makeBuilderContextCE32back to summary
protected static int makeBuilderContextCE32(int index)
optimizeback to summary
pack-priv void optimize(UnicodeSet set)
setDigitTagsback to summary
protected void setDigitTags()
setLeadSurrogatesback to summary
protected void setLeadSurrogates()
suppressContractionsback to summary
pack-priv void suppressContractions(UnicodeSet set)
com.ibm.icu.impl.coll back to summary

pack-priv Interface CollationDataBuilder.CEModifier

Known Direct Implementers
com.ibm.icu.impl.coll.CollationBuilder.CEFinalizer

Collation element modifier. Interface class for a modifier that changes a tailoring builder's temporary CEs to final CEs. Called for every non-special CE32 and every expansion CE.

Method Summary

Modifier and TypeMethod and Description
public long
modifyCE(long ce)

Returns a new CE to replace the input CE, or else Collation.NO_CE.

public long
modifyCE32(int ce32)

Returns a new CE to replace the non-special input CE32, or else Collation.NO_CE.

Method Detail

modifyCEback to summary
public long modifyCE(long ce)

Returns a new CE to replace the input CE, or else Collation.NO_CE.

modifyCE32back to summary
public long modifyCE32(int ce32)

Returns a new CE to replace the non-special input CE32, or else Collation.NO_CE.

com.ibm.icu.impl.coll back to summary

private final Class CollationDataBuilder.ConditionalCE32

extends Object
Class Inheritance

Build-time context and CE32 for a code point. If a code point has contextual mappings, then the default (no-context) mapping and all conditional mappings are stored in a singly-linked list of ConditionalCE32, sorted by context strings. Context strings sort by prefix length, then by prefix, then by contraction suffix. Context strings must be unique and in ascending order.

Field Summary

Modifier and TypeField and Description
pack-priv int
builtCE32

CE32 for the built contexts.

pack-priv int
ce32

CE32 for the code point and its context.

pack-priv String
context

"\0" for the first entry for any code point, with its default CE32.

pack-priv int
defaultCE32

Default CE32 for all contexts with this same prefix.

pack-priv int
era

The "era" of building intermediate contexts when the above builtCE32 was set.

pack-priv int
next

Index of the next ConditionalCE32.

Constructor Summary

AccessConstructor and Description
pack-priv
ConditionalCE32(String ct, int ce)

Method Summary

Modifier and TypeMethod and Description
pack-priv boolean
pack-priv int
Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

builtCE32back to summary
pack-priv int builtCE32

CE32 for the built contexts. When fetching CEs from the builder, the contexts are built into their runtime form so that the normal collation implementation can process them. The result is cached in the list head. It is reset when the contexts are modified. All of these builtCE32 are invalidated by clearContexts(), via incrementing the contextsEra.

ce32back to summary
pack-priv int ce32

CE32 for the code point and its context. Can be special (e.g., for an expansion) but not contextual (prefix or contraction tag).

contextback to summary
pack-priv String context

"\0" for the first entry for any code point, with its default CE32. Otherwise one unit with the length of the prefix string, then the prefix string, then the contraction suffix.

defaultCE32back to summary
pack-priv int defaultCE32

Default CE32 for all contexts with this same prefix. Initially NO_CE32. Set only while building runtime data structures, and only on one of the nodes of a sub-list with the same prefix.

eraback to summary
pack-priv int era

The "era" of building intermediate contexts when the above builtCE32 was set. When the array of cached, temporary contexts overflows, then clearContexts() removes them all and invalidates the builtCE32 that used to point to built tries.

nextback to summary
pack-priv int next

Index of the next ConditionalCE32. Negative for the end of the list.

Constructor Detail

ConditionalCE32back to summary
pack-priv ConditionalCE32(String ct, int ce)

Method Detail

hasContextback to summary
pack-priv boolean hasContext()
prefixLengthback to summary
pack-priv int prefixLength()
com.ibm.icu.impl.coll back to summary

private final Class CollationDataBuilder.CopyHelper

extends Object
Class Inheritance

Field Summary

Modifier and TypeField and Description
pack-priv CollationDataBuilder
pack-priv long[]
pack-priv CollationDataBuilder.CEModifier
pack-priv CollationDataBuilder

Constructor Summary

AccessConstructor and Description
pack-priv

Method Summary

Modifier and TypeMethod and Description
pack-priv int
copyCE32(int ce32)

pack-priv void
copyRangeCE32(int start, int end, int ce32)

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

destback to summary
pack-priv CollationDataBuilder dest
modifiedCEsback to summary
pack-priv long[] modifiedCEs
modifierback to summary
pack-priv CollationDataBuilder.CEModifier modifier
srcback to summary
pack-priv CollationDataBuilder src

Constructor Detail

CopyHelperback to summary
pack-priv CopyHelper(CollationDataBuilder s, CollationDataBuilder d, CollationDataBuilder.CEModifier m)

Method Detail

copyCE32back to summary
pack-priv int copyCE32(int ce32)
copyRangeCE32back to summary
pack-priv void copyRangeCE32(int start, int end, int ce32)
com.ibm.icu.impl.coll back to summary

private final Class CollationDataBuilder.DataBuilderCollationIterator

extends CollationIterator
Class Inheritance

Build-time collation element and character iterator. Uses the runtime CollationIterator for fetching CEs for a string but reads from the builder's unfinished data structures. In particular, this class reads from the unfinished trie and has to avoid CollationIterator.nextCE() and redirect other calls to data.getCE32() and data.getCE32FromSupplementary(). We do this so that we need not implement the collation algorithm again for the builder and make it behave exactly like the runtime code. That would be more difficult to test and maintain than this indirection. Some CE32 tags (for example, the DIGIT_TAG) do not occur in the builder data, so the data accesses from those code paths need not be modified. This class iterates directly over whole code points so that the CollationIterator does not need the finished trie for handling the LEAD_SURROGATE_TAG.

Field Summary

Modifier and TypeField and Description
protected final CollationDataBuilder
protected final CollationData
protected final int[]
protected int
protected CharSequence
s

Inherited from com.ibm.icu.impl.coll.CollationIterator:
dataNO_CP_AND_CE32trie

Constructor Summary

AccessConstructor and Description
pack-priv

Method Summary

Modifier and TypeMethod and Description
protected void
pack-priv int
fetchCEs(CharSequence str, int start, long[] ces, int cesLength)

protected void
protected int
protected int
getDataCE32(int c)

Overrides com.ibm.icu.impl.coll.CollationIterator.getDataCE32.

Returns the CE32 from the data trie.
public int
public int
nextCodePoint()

Implements abstract com.ibm.icu.impl.coll.CollationIterator.nextCodePoint.

Returns the next code point (with post-increment).
public int
previousCodePoint()

Implements abstract com.ibm.icu.impl.coll.CollationIterator.previousCodePoint.

Returns the previous code point (with pre-decrement).
public void
resetToOffset(int newOffset)

Implements abstract com.ibm.icu.impl.coll.CollationIterator.resetToOffset.

Resets the iterator state and sets the position to the specified offset.
Inherited from com.ibm.icu.impl.coll.CollationIterator:
appendCEsFromCE32clearCEsclearCEsIfNoneRemainingequalsfetchCEsforbidSurrogateCodePointsgetCEgetCEsgetCEsLengthhandleGetTrailSurrogatehandleNextCE32hashCodeisLeadSurrogateisTrailSurrogatemakeCodePointAndCE32PairnextCEpreviousCEresetresetsetCurrentCE

Field Detail

builderback to summary
protected final CollationDataBuilder builder
builderDataback to summary
protected final CollationData builderData
jamoCE32sback to summary
protected final int[] jamoCE32s
posback to summary
protected int pos
sback to summary
protected CharSequence s

Constructor Detail

DataBuilderCollationIteratorback to summary
pack-priv DataBuilderCollationIterator(CollationDataBuilder b, CollationData newData)

Method Detail

backwardNumCodePointsback to summary
protected void backwardNumCodePoints(int num)

Implements abstract com.ibm.icu.impl.coll.CollationIterator.backwardNumCodePoints.

Annotations
@Override
fetchCEsback to summary
pack-priv int fetchCEs(CharSequence str, int start, long[] ces, int cesLength)
forwardNumCodePointsback to summary
protected void forwardNumCodePoints(int num)

Implements abstract com.ibm.icu.impl.coll.CollationIterator.forwardNumCodePoints.

Annotations
@Override
getCE32FromBuilderDataback to summary
protected int getCE32FromBuilderData(int ce32)

Overrides com.ibm.icu.impl.coll.CollationIterator.getCE32FromBuilderData.

Annotations
@Override
getDataCE32back to summary
protected int getDataCE32(int c)

Overrides com.ibm.icu.impl.coll.CollationIterator.getDataCE32.

Doc from com.ibm.icu.impl.coll.CollationIterator.getDataCE32.

Returns the CE32 from the data trie. Normally the same as data.getCE32(), but overridden in the builder. Call this only when the faster data.getCE32() cannot be used.

Annotations
@Override
getOffsetback to summary
public int getOffset()

Implements abstract com.ibm.icu.impl.coll.CollationIterator.getOffset.

Annotations
@Override
nextCodePointback to summary
public int nextCodePoint()

Implements abstract com.ibm.icu.impl.coll.CollationIterator.nextCodePoint.

Doc from com.ibm.icu.impl.coll.CollationIterator.nextCodePoint.

Returns the next code point (with post-increment). Public for identical-level comparison and for testing.

Annotations
@Override
previousCodePointback to summary
public int previousCodePoint()

Implements abstract com.ibm.icu.impl.coll.CollationIterator.previousCodePoint.

Doc from com.ibm.icu.impl.coll.CollationIterator.previousCodePoint.

Returns the previous code point (with pre-decrement). Public for identical-level comparison and for testing.

Annotations
@Override
resetToOffsetback to summary
public void resetToOffset(int newOffset)

Implements abstract com.ibm.icu.impl.coll.CollationIterator.resetToOffset.

Doc from com.ibm.icu.impl.coll.CollationIterator.resetToOffset.

Resets the iterator state and sets the position to the specified offset. Subclasses must implement, and must call the parent class method, or CollationIterator.reset().

Annotations
@Override