Top Description Inners Fields Constructors Methods
com.ibm.icu.impl

public final Class Normalizer2Impl

extends Object
Class Inheritance
Imports
java.io.IOException, java.nio.ByteBuffer, java.util.ArrayList, com.ibm.icu.text.UTF16, .UnicodeSet, com.ibm.icu.util.CodePointMap, .CodePointTrie, .ICUUncheckedIOException, .MutableCodePointTrie, .VersionInfo
Low-level implementation of the Unicode Normalization Algorithm. For the data structure and details see the documentation at the end of C++ normalizer2impl.h and in the design doc at https://icu.unicode.org/design/normalization/custom

Nested and Inner Type Summary

Modifier and TypeClass and Description
public static class
Normalizer2Impl.Hangul

private static class
Normalizer2Impl.IsAcceptable

public static class
Normalizer2Impl.ReorderingBuffer

Writable buffer that takes care of canonical ordering.

public static class
Normalizer2Impl.UTF16Plus

Field Summary

Modifier and TypeField and Description
private static final int
CANON_HAS_COMPOSITIONS

private static final int
CANON_HAS_SET

private static final int
CANON_NOT_SEGMENT_STARTER

private static final int
CANON_VALUE_MASK

private CodePointTrie
canonIterData

private ArrayList<UnicodeSet>
canonStartSets

private int
centerNoNoDelta

public static final int
COMP_1_LAST_TUPLE

public static final int
COMP_1_TRAIL_LIMIT

public static final int
COMP_1_TRAIL_MASK

public static final int
COMP_1_TRAIL_SHIFT

public static final int
COMP_1_TRIPLE

public static final int
COMP_2_TRAIL_MASK

public static final int
COMP_2_TRAIL_SHIFT

private static final int
DATA_FORMAT

private VersionInfo
dataVersion

public static final int
DELTA_SHIFT

public static final int
DELTA_TCCC_0

public static final int
DELTA_TCCC_1

public static final int
DELTA_TCCC_GT_1

public static final int
DELTA_TCCC_MASK

private String
extraData

public static final int
HAS_COMP_BOUNDARY_AFTER

public static final int
INERT

private static final Normalizer2Impl.IsAcceptable
IS_ACCEPTABLE

public static final int
IX_COUNT

public static final int
IX_EXTRA_DATA_OFFSET

public static final int
IX_LIMIT_NO_NO

public static final int
IX_MIN_COMP_NO_MAYBE_CP

public static final int
IX_MIN_DECOMP_NO_CP

public static final int
IX_MIN_LCCC_CP

public static final int
IX_MIN_MAYBE_YES

public static final int
IX_MIN_NO_NO

Mappings are comp-normalized.

public static final int
IX_MIN_NO_NO_COMP_BOUNDARY_BEFORE

Mappings are not comp-normalized but have a comp boundary before.

public static final int
IX_MIN_NO_NO_COMP_NO_MAYBE_CC

Mappings do not have a comp boundary before.

public static final int
IX_MIN_NO_NO_EMPTY

Mappings to the empty string.

public static final int
IX_MIN_YES_NO

Mappings & compositions in [minYesNo..minYesNoMappingsOnly[.

public static final int
IX_MIN_YES_NO_MAPPINGS_ONLY

Mappings only in [minYesNoMappingsOnly..minNoNo[.

public static final int
IX_NORM_TRIE_OFFSET

public static final int
IX_RESERVED3_OFFSET

public static final int
IX_SMALL_FCD_OFFSET

public static final int
IX_TOTAL_SIZE

public static final int
JAMO_L

public static final int
JAMO_VT

private int
limitNoNo

public static final int
MAPPING_HAS_CCC_LCCC_WORD

public static final int
MAPPING_HAS_RAW_MAPPING

public static final int
MAPPING_LENGTH_MASK

public static final int
MAX_DELTA

private String
maybeYesCompositions

public static final int
MIN_NORMAL_MAYBE_YES

public static final int
MIN_YES_YES_WITH_CC

private int
minCompNoMaybeCP

private int
minDecompNoCP

private int
minLcccCP

private int
minMaybeYes

private int
minNoNo

private int
minNoNoCompBoundaryBefore

private int
minNoNoCompNoMaybeCC

private int
minNoNoEmpty

private int
minYesNo

private int
minYesNoMappingsOnly

private CodePointTrie.Fast16
normTrie

public static final int
OFFSET_SHIFT

private static final CodePointMap.ValueFilter
segmentStarterMapper

private byte[]
smallFCD

Constructor Summary

AccessConstructor and Description
public
Normalizer2Impl()

Method Summary

Modifier and TypeMethod and Description
public void
addCanonIterPropertyStarts(UnicodeSet set)

private void
addComposites(int
some character's compositions list
list, UnicodeSet
recursively receives the composites from these compositions
set)

public void
addLcccChars(UnicodeSet set)

public void
addPropertyStarts(UnicodeSet set)

private void
addToStartSet(MutableCodePointTrie mutableTrie, int origin, int decompLead)

private static int
combine(String compositions, int list, int trail)

Finds the recomposition result for a forward-combining "lead" character, specified with a pointer to its compositions list, and a backward-combining "trail" character.

public boolean
compose(CharSequence s, int src, int limit, boolean onlyContiguous, boolean doCompose, Normalizer2Impl.ReorderingBuffer buffer)

public void
composeAndAppend(CharSequence s, boolean doCompose, boolean onlyContiguous, Normalizer2Impl.ReorderingBuffer buffer)

public int
composePair(int a, int b)

public int

Returns:

bits 31..1: spanQuickCheckYes (==s.length() if "yes") and bit 0: set if "maybe"; otherwise, if the span length<s.length() then the quick check result is "no"
composeQuickCheck(CharSequence s, int src, int limit, boolean onlyContiguous, boolean doSpan)

Very similar to compose(): Make the same changes in both places if relevant.

public Appendable
decompose(CharSequence s, StringBuilder dest)

public void
decompose(CharSequence s, int src, int limit, StringBuilder dest, int destLengthEstimate)

Decomposes s[src, limit[ and writes the result to dest.

public int
decompose(CharSequence s, int src, int limit, Normalizer2Impl.ReorderingBuffer buffer)

private void
decompose(int c, int norm16, Normalizer2Impl.ReorderingBuffer buffer)

public void
decomposeAndAppend(CharSequence s, boolean doDecompose, Normalizer2Impl.ReorderingBuffer buffer)

private int
decomposeShort(CharSequence s, int src, int limit, boolean stopAtCompBoundary, boolean onlyContiguous, Normalizer2Impl.ReorderingBuffer buffer)

public synchronized Normalizer2Impl

Returns:

this
ensureCanonIterData()

Builds the canonical-iterator data for this instance.

private int
findNextCompBoundary(CharSequence s, int p, int limit, boolean onlyContiguous)

private int
findNextFCDBoundary(CharSequence s, int p, int limit)

private int
findPreviousCompBoundary(CharSequence s, int p, boolean onlyContiguous)

private int
findPreviousFCDBoundary(CharSequence s, int p)

public boolean

Returns:

true if there are characters whose decomposition starts with c.
getCanonStartSet(int
A Unicode code point.
c, UnicodeSet
A UnicodeSet to receive the characters whose decompositions start with c, if there are any.
set)

Returns true if there are characters whose decomposition starts with c.

public int
getCC(int norm16)

private int
getCCFromNoNo(int norm16)

public static int
getCCFromNormalYesOrMaybe(int norm16)

public static int
getCCFromYesOrMaybe(int norm16)

public int
getCCFromYesOrMaybeCP(int c)

private int

Returns:

index into maybeYesCompositions
getCompositionsList(int
code point must have compositions
norm16)

private int

Returns:

index into maybeYesCompositions
getCompositionsListForComposite(int norm16)

private int

Returns:

index into maybeYesCompositions, or -1
getCompositionsListForDecompYes(int norm16)

private int
getCompositionsListForMaybe(int norm16)

public int
getCompQuickCheck(int norm16)

public String

Returns:

c's decomposition, if it has one; returns null if it does not have a decomposition
getDecomposition(int
code point
c)

Gets the decomposition for one code point.

public int

Returns:

The lccc(c) in bits 15..8 and tccc(c) in bits 7..0.
getFCD16(int
A Unicode code point.
c)

Returns the FCD data for code point c.

public int
getFCD16FromNormData(int c)

Gets the FCD value from the regular normalization data.

public int
getNorm16(int c)

private int
getPreviousTrailCC(CharSequence s, int start, int p)

public String

Returns:

c's raw decomposition, if it has one; returns null if it does not have a decomposition
getRawDecomposition(int
code point
c)

Gets the raw decomposition for one code point.

public int
getRawNorm16(int c)

pack-priv int
getTrailCCFromCompYesAndZeroCC(int norm16)

private int
hangulLVT()

public boolean
hasCompBoundaryAfter(int c, boolean onlyContiguous)

private boolean
hasCompBoundaryAfter(CharSequence s, int start, int p, boolean onlyContiguous)

public boolean
hasCompBoundaryBefore(int c)

private boolean
hasCompBoundaryBefore(int c, int norm16)

Does c have a composition boundary before it?

private boolean
hasCompBoundaryBefore(CharSequence s, int src, int limit)

public boolean
hasDecompBoundaryAfter(int c)

public boolean
hasDecompBoundaryBefore(int c)

public boolean
hasFCDBoundaryAfter(int c)

public boolean
hasFCDBoundaryBefore(int c)

public boolean
isAlgorithmicNoNo(int norm16)

public boolean

Returns:

true if c starts a canonical-iterator string segment.
isCanonSegmentStarter(int
A Unicode code point.
c)

Returns true if code point c starts a canonical-iterator string segment.

public boolean
isCompInert(int c, boolean onlyContiguous)

public boolean
isCompNo(int norm16)

private boolean
isCompYesAndZeroCC(int norm16)

public boolean
isDecompInert(int c)

private boolean
isDecompNoAlgorithmic(int norm16)

public boolean
isDecompYes(int norm16)

private boolean
isDecompYesAndZeroCC(int norm16)

public boolean
isFCDInert(int c)

private boolean
isHangulLV(int norm16)

private boolean
isHangulLVT(int norm16)

private static boolean
isInert(int norm16)

private static boolean
isJamoL(int norm16)

private static boolean
isJamoVT(int norm16)

private boolean
isMaybe(int norm16)

private boolean
isMaybeOrNonZeroCC(int norm16)

private boolean
isMostDecompYesAndZeroCC(int norm16)

A little faster and simpler than isDecompYesAndZeroCC() but does not include the MaybeYes which combine-forward and have ccc=0.

private boolean
isTrailCC01ForCompBoundaryAfter(int norm16)

For FCC: Given norm16 HAS_COMP_BOUNDARY_AFTER, does it have tccc<=1?

public Normalizer2Impl
load(ByteBuffer bytes)

public Normalizer2Impl
load(String name)

public int
makeFCD(CharSequence s, int src, int limit, Normalizer2Impl.ReorderingBuffer buffer)

public void
makeFCDAndAppend(CharSequence s, boolean doMakeFCD, Normalizer2Impl.ReorderingBuffer buffer)

private int
mapAlgorithmic(int c, int norm16)

private boolean
norm16HasCompBoundaryAfter(int norm16, boolean onlyContiguous)

private boolean
norm16HasCompBoundaryBefore(int norm16)

public boolean
norm16HasDecompBoundaryAfter(int norm16)

public boolean
norm16HasDecompBoundaryBefore(int norm16)

private void
recompose(Normalizer2Impl.ReorderingBuffer buffer, int recomposeStartIndex, boolean onlyContiguous)

public boolean
singleLeadMightHaveNonZeroFCD16(int lead)

Returns true if the single-or-lead code unit c might have non-zero FCD data.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait