Modifier and Type	Class and Description
pack-priv class	RuleBasedBreakIterator.BreakCache
pack-priv class	RuleBasedBreakIterator.DictionaryCache DictionaryCache stores the boundaries obtained from a run of dictionary characters.

Modifier and Type	Field and Description
private RuleBasedBreakIterator.BreakCache	fBreakCache Cache of previously determined boundary positions.
private List<LanguageBreakEngine>	fBreakEngines List of all known break engines.
public static final String	fDebugEnv Control debug, trace and dump options.
private RuleBasedBreakIterator.DictionaryCache	fDictionaryCache
private int	fDictionaryCharCount Counter for the number of characters encountered with the "dictionary" flag set.
private boolean	fDone True when iteration has run off the end, and iterator functions should return UBRK_DONE.
private int[]	fLookAheadMatches Array of look-ahead tentative results.
private boolean	fPhraseBreaking Flag used to indicate if phrase breaking is required.
private int	fPosition The iteration state - current position, rule status for the current position, and whether the iterator ran off the end, yielding UBRK_DONE.
public RBBIDataWrapper	fRData The rule data for this BreakIterator instance.
private int	fRuleStatusIndex Index of the Rule {tag} values for the most recent match.
private CharacterIterator	fText The character iterator through which this BreakIterator accesses the text.
private static final List<LanguageBreakEngine>	gAllBreakEngines List of all known break engines, common for all break iterators.
private static final UnhandledBreakEngine	gUnhandledBreakEngine The "default" break engine - just skips over ranges of dictionary words, producing no breaks.
private static final String	RBBI_DEBUG_ARG ICU debug argument name for RBBI
private static final int	RBBI_END
private static final int	RBBI_RUN
private static final int	RBBI_START
private static final int	START_STATE
private static final int	STOP_STATE
private static final boolean	TRACE Debugging flag.

Access	Constructor and Description
private	RuleBasedBreakIterator() private constructor
public	RuleBasedBreakIterator(String The break rules to be used. rules) Construct a RuleBasedBreakIterator from a set of rules supplied as a string.

Method Summary

Modifier and Type	Method and Description
protected static final void	checkOffset(int offset, CharacterIterator text) Throw IllegalArgumentException unless begin <= offset < end.
private static int	Returns: the resulting index, possibly pinned or adjusted. CISetIndex32(CharacterIterator A CharacterIterator to set ci, int the index to set index) Set the index of a CharacterIterator.
public Object	Returns: A newly-constructed RuleBasedBreakIterator with the same behavior as this one. clone() Overrides com.ibm.icu.text.BreakIterator.clone. Clones this iterator.
public static void	compileRules(String The source form of the break rules rules, OutputStream An output stream to receive the compiled rules. ruleBinary) Compile a set of source break rules into the binary state tables used by the break iterator engine.
public int	Returns: The current iteration position. current() Implements abstract com.ibm.icu.text.BreakIterator.current. Returns the current iteration position.
public void	dump(PrintStream out) Deprecated This API is ICU internal only. Dump the contents of the state table and character classes for this break iterator.
public boolean	equals(Object the reference object with which to compare. that) Overrides java.lang.Object.equals. Returns true if both BreakIterators are of the same class, have the same rules, and iterate over the same text.
public int	Returns: The offset of the beginning of the text. first() Implements abstract com.ibm.icu.text.BreakIterator.first. Sets the current iteration position to the beginning of the text.
public int	Returns: The position of the first break after the current position. following(int The position from which to begin searching for a break position. startPos) Implements abstract com.ibm.icu.text.BreakIterator.following. Sets the iterator to refer to the first boundary position following the specified position.
public static RuleBasedBreakIterator	getInstanceFromCompiledRules(InputStream an input stream supplying the compiled binary rules. is) Create a break iterator from a precompiled set of break rules.
pack-priv static RuleBasedBreakIterator	getInstanceFromCompiledRules(ByteBuffer a buffer supplying the compiled binary rules. bytes, boolean a flag indicating if phrase breaking is required. phraseBreaking) This factory method doesn't have an access modifier; it is only accessible in the same package.
public static RuleBasedBreakIterator	getInstanceFromCompiledRules(ByteBuffer a buffer supplying the compiled binary rules. bytes) Deprecated This API is ICU internal only. Create a break iterator from a precompiled set of break rules.
private LanguageBreakEngine	getLanguageBreakEngine(int c)
public int	Returns: the status from the break rule that determined the boundary at the current iteration position. getRuleStatus() Overrides com.ibm.icu.text.BreakIterator.getRuleStatus. Return the status tag from the break rule that determined the boundary at the current iteration position.
public int	Returns: The number of rule status values from the rules that determined the boundary at the current iteration position. In the event that the array is too small, the return value is the total number of status values that were available, not the reduced number that were actually returned. getRuleStatusVec(int[] an array to be filled in with the status values. fillInArray) Overrides com.ibm.icu.text.BreakIterator.getRuleStatusVec. Get the status (tag) values from the break rule(s) that determined the boundary at the current iteration position.
public CharacterIterator	Returns: An iterator over the text being analyzed. getText() Implements abstract com.ibm.icu.text.BreakIterator.getText. Returns a CharacterIterator over the text being analyzed.
private int	Returns: the new iterator position A note on supplementary characters and the position of underlying Java CharacterIterator: Normally, a character iterator is positioned at the char most recently returned by next(). Within this function, when a supplementary char is being processed, the char iterator is left sitting on the trail surrogate, in the middle of the code point. This is different from everywhere else, where an iterator always points at the lead surrogate of a supplementary. handleNext() The State Machine Engine for moving forward is here.
private int	handleSafePrevious(int the position in the input text to begin the iteration. fromPosition) Iterate backwards from an arbitrary position in the input text using the Safe Reverse rules.
public int	Returns: A hash code hashCode() Overrides java.lang.Object.hashCode. Compute a hashcode for this BreakIterator
public boolean	Returns: True if "offset" is a boundary position. isBoundary(int the offset to check. offset) Overrides com.ibm.icu.text.BreakIterator.isBoundary. Returns true if the specified position is a boundary position.
public int	Returns: The text's past-the-end offset. last() Implements abstract com.ibm.icu.text.BreakIterator.last. Sets the current iteration position to the end of the text.
public int	Returns: The character offset of the boundary position n boundaries away from the current one. next(int The number of steps to move. The sign indicates the direction (negative is backwards, and positive is forwards). n) Implements abstract com.ibm.icu.text.BreakIterator.next. Advances the iterator either forward or backward the specified number of steps.
public int	Returns: The position of the first boundary after this one. next() Implements abstract com.ibm.icu.text.BreakIterator.next. Advances the iterator to the next boundary position.
public int	Returns: The position of the last boundary before the starting position. preceding(int The position to begin searching for a break from. offset) Overrides com.ibm.icu.text.BreakIterator.preceding. Sets the iterator to refer to the last boundary position before the specified position.
public int	Returns: The position of the boundary position immediately preceding the starting position. previous() Implements abstract com.ibm.icu.text.BreakIterator.previous. Moves the iterator backwards, to the boundary preceding the current one.
public void	setText(CharacterIterator An iterator over the text to analyze. newText) Implements abstract com.ibm.icu.text.BreakIterator.setText. Set the iterator to analyze a new piece of text.
public String	toString() Overrides java.lang.Object.toString. Returns the description (rules) used to create this iterator.

Field Detail

fBreakCache	back to summary
private RuleBasedBreakIterator.BreakCache fBreakCache Cache of previously determined boundary positions.

fBreakEngines	back to summary
private List<LanguageBreakEngine> fBreakEngines List of all known break engines. Similar to gAllBreakEngines, but local to a break iterator, allowing it to be used without synchronization.

fDebugEnv	back to summary
public static final String fDebugEnv Deprecated This API is ICU internal only. Control debug, trace and dump options. Annotations @Deprecated ICU Status Internal ICU component.

fDictionaryCache	back to summary
private RuleBasedBreakIterator.DictionaryCache fDictionaryCache

fDictionaryCharCount	back to summary
private int fDictionaryCharCount Counter for the number of characters encountered with the "dictionary" flag set. Normal RBBI iterators don't use it, although the code for updating it is live. Dictionary Based break iterators (a subclass of us) access this field directly. ICU Status Internal ICU component.

fDictionaryCharCount

back to summary

private int fDictionaryCharCount

Counter for the number of characters encountered with the "dictionary" flag set. Normal RBBI iterators don't use it, although the code for updating it is live. Dictionary Based break iterators (a subclass of us) access this field directly.

ICU Status: Internal ICU component.

fDone	back to summary
private boolean fDone True when iteration has run off the end, and iterator functions should return UBRK_DONE.

fLookAheadMatches	back to summary
private int[] fLookAheadMatches Array of look-ahead tentative results.

fPhraseBreaking	back to summary
private boolean fPhraseBreaking Flag used to indicate if phrase breaking is required.

fPosition	back to summary
private int fPosition The iteration state - current position, rule status for the current position, and whether the iterator ran off the end, yielding UBRK_DONE. Current position is pinned to be 0 < position <= text.length. Current position is always set to a boundary. The current position of the iterator. Pinned, 0 < fPosition <= text.length. Never has the value UBRK_DONE (-1).

fRData	back to summary
public RBBIDataWrapper fRData Deprecated This API is ICU internal only. The rule data for this BreakIterator instance. Not intended for public use. Declared public for testing purposes only. Annotations @Deprecated ICU Status Internal ICU component.

fRuleStatusIndex	back to summary
private int fRuleStatusIndex Index of the Rule {tag} values for the most recent match.

fText	back to summary
private CharacterIterator fText The character iterator through which this BreakIterator accesses the text.

gAllBreakEngines	back to summary
private static final List<LanguageBreakEngine> gAllBreakEngines List of all known break engines, common for all break iterators. Lazily updated as break engines are needed, because instantiation of break engines is expensive. Because gAllBreakEngines can be referenced concurrently from different BreakIterator instances, all access is synchronized.

gUnhandledBreakEngine	back to summary
private static final UnhandledBreakEngine gUnhandledBreakEngine The "default" break engine - just skips over ranges of dictionary words, producing no breaks. Should only be used if characters need to be handled by a dictionary but we have no dictionary implementation for them. Only one instance; shared by all break iterators.

RBBI_DEBUG_ARG	back to summary
private static final String RBBI_DEBUG_ARG ICU debug argument name for RBBI

RBBI_END	back to summary
private static final int RBBI_END

RBBI_RUN	back to summary
private static final int RBBI_RUN

RBBI_START	back to summary
private static final int RBBI_START

START_STATE	back to summary
private static final int START_STATE

STOP_STATE	back to summary
private static final int STOP_STATE

TRACE	back to summary
private static final boolean TRACE Debugging flag. Trace operation of state machine when true.

Constructor Detail

RuleBasedBreakIterator	back to summary
private RuleBasedBreakIterator() private constructor

RuleBasedBreakIterator	back to summary
public RuleBasedBreakIterator(String rules) Construct a RuleBasedBreakIterator from a set of rules supplied as a string. Parameters rules:String The break rules to be used. ICU Status Stable since ICU 2.2.

RuleBasedBreakIterator

back to summary

public RuleBasedBreakIterator(String rules)

Construct a RuleBasedBreakIterator from a set of rules supplied as a string.

Parameters

rules:String: The break rules to be used.

ICU Status

Stable since ICU 2.2.

Method Detail

checkOffset	back to summary
protected static final void checkOffset(int offset, CharacterIterator text) Throw IllegalArgumentException unless begin <= offset < end. ICU Status Stable since ICU 2.0.

checkOffset

back to summary

protected static final void checkOffset(int offset, CharacterIterator text)

Throw IllegalArgumentException unless begin <= offset < end.

ICU Status: Stable since ICU 2.0.

CISetIndex32	back to summary
private static int CISetIndex32(CharacterIterator ci, int index) Set the index of a CharacterIterator. Pin the index to the valid range range of BeginIndex <= index <= EndIndex. If the index points to a trail surrogate of a supplementary character, adjust it to the start (lead surrogate) index. Parameters ci:CharacterIterator A CharacterIterator to set index:int the index to set Returns:int the resulting index, possibly pinned or adjusted.

CISetIndex32

back to summary

private static int CISetIndex32(CharacterIterator ci, int index)

Set the index of a CharacterIterator. Pin the index to the valid range range of BeginIndex <= index <= EndIndex. If the index points to a trail surrogate of a supplementary character, adjust it to the start (lead surrogate) index.

Parameters

ci:CharacterIterator: A CharacterIterator to set
index:int: the index to set

Returns:int

the resulting index, possibly pinned or adjusted.

clone	back to summary
public Object clone() Overrides com.ibm.icu.text.BreakIterator.clone. Clones this iterator. Returns:Object A newly-constructed RuleBasedBreakIterator with the same behavior as this one. Annotations @Override ICU Status Stable since ICU 2.0.

clone

back to summary

public Object clone()

Overrides com.ibm.icu.text.BreakIterator.clone.

Clones this iterator.

Returns:Object

A newly-constructed RuleBasedBreakIterator with the same behavior as this one.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

compileRules back to summary

compileRules	back to summary
public static void compileRules(String rules, OutputStream ruleBinary) throws IOException Compile a set of source break rules into the binary state tables used by the break iterator engine. Creating a break iterator from precompiled rules is much faster than creating one from source rules. Binary break rules are not guaranteed to be compatible between different versions of ICU. Parameters rules:String The source form of the break rules ruleBinary:OutputStream An output stream to receive the compiled rules. Exceptions IOException: If there is an error writing the output. ICU Status Stable since ICU 4.8. See Also `getInstanceFromCompiledRules(InputStream)`

public static void compileRules(String rules, OutputStream ruleBinary) throws IOException

Compile a set of source break rules into the binary state tables used by the break iterator engine. Creating a break iterator from precompiled rules is much faster than creating one from source rules. Binary break rules are not guaranteed to be compatible between different versions of ICU.

Parameters

rules:String: The source form of the break rules
ruleBinary:OutputStream: An output stream to receive the compiled rules.

Exceptions

IOException:: If there is an error writing the output.

ICU Status

Stable since ICU 4.8.

current	back to summary
public int current() Implements abstract com.ibm.icu.text.BreakIterator.current. Returns the current iteration position. Note that DONE is never returned from this function; if iteration has run to the end of a string, current() will return the length of the string while next() will return BreakIterator.DONE). Returns:int The current iteration position. Annotations @Override ICU Status Stable since ICU 2.0.

current

back to summary

public int current()

Implements abstract com.ibm.icu.text.BreakIterator.current.

Returns the current iteration position. Note that DONE is never returned from this function; if iteration has run to the end of a string, current() will return the length of the string while next() will return BreakIterator.DONE).

Returns:int

The current iteration position.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

dump	back to summary
public void dump(PrintStream out) Deprecated This API is ICU internal only. Dump the contents of the state table and character classes for this break iterator. For debugging only. Annotations @Deprecated ICU Status Internal ICU component.

dump

back to summary

public void dump(PrintStream out)

Deprecated

This API is ICU internal only.

Dump the contents of the state table and character classes for this break iterator. For debugging only.

Annotations

@Deprecated

ICU Status

Internal ICU component.

equals back to summary

equals	back to summary
public boolean equals(Object that) Overrides java.lang.Object.equals. Returns true if both BreakIterators are of the same class, have the same rules, and iterate over the same text. Parameters that:Object Doc from java.lang.Object.equals. the reference object with which to compare. Returns:boolean Doc from java.lang.Object.equals. `true` if this object is the same as the obj argument; `false` otherwise. Annotations @Override ICU Status Stable since ICU 2.0.

public boolean equals(Object that)

Overrides java.lang.Object.equals.

Returns true if both BreakIterators are of the same class, have the same rules, and iterate over the same text.

Parameters

that:Object

Doc from java.lang.Object.equals.

the reference object with which to compare.

Returns:boolean

Doc from java.lang.Object.equals.

true if this object is the same as the obj argument; false otherwise.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

first	back to summary
public int first() Implements abstract com.ibm.icu.text.BreakIterator.first. Sets the current iteration position to the beginning of the text. (i.e., the CharacterIterator's starting offset). Returns:int The offset of the beginning of the text. Annotations @Override ICU Status Stable since ICU 2.0.

first

back to summary

public int first()

Implements abstract com.ibm.icu.text.BreakIterator.first.

Sets the current iteration position to the beginning of the text. (i.e., the CharacterIterator's starting offset).

Returns:int

The offset of the beginning of the text.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

following	back to summary
public int following(int startPos) Implements abstract com.ibm.icu.text.BreakIterator.following. Sets the iterator to refer to the first boundary position following the specified position. Parameters startPos:int The position from which to begin searching for a break position. Returns:int The position of the first break after the current position. Annotations @Override ICU Status Stable since ICU 2.0.

following

back to summary

public int following(int startPos)

Implements abstract com.ibm.icu.text.BreakIterator.following.

Sets the iterator to refer to the first boundary position following the specified position.

Parameters

startPos:int: The position from which to begin searching for a break position.

Returns:int

The position of the first break after the current position.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

getInstanceFromCompiledRules back to summary

getInstanceFromCompiledRules	back to summary
public static RuleBasedBreakIterator getInstanceFromCompiledRules(InputStream is) throws IOException Create a break iterator from a precompiled set of break rules. Creating a break iterator from the binary rules is much faster than creating one from source rules. The binary rules are generated by the RuleBasedBreakIterator.compileRules() function. Binary break iterator rules are not guaranteed to be compatible between different versions of ICU. Parameters is:InputStream an input stream supplying the compiled binary rules. Exceptions IOException: if there is an error while reading the rules from the InputStream. ICU Status Stable since ICU 4.8. See Also `compileRules(String, OutputStream)`

public static RuleBasedBreakIterator getInstanceFromCompiledRules(InputStream is) throws IOException

Create a break iterator from a precompiled set of break rules. Creating a break iterator from the binary rules is much faster than creating one from source rules. The binary rules are generated by the RuleBasedBreakIterator.compileRules() function. Binary break iterator rules are not guaranteed to be compatible between different versions of ICU.

Parameters

is:InputStream: an input stream supplying the compiled binary rules.

Exceptions

IOException:: if there is an error while reading the rules from the InputStream.

ICU Status

Stable since ICU 4.8.

See Also

compileRules(String, OutputStream)

getInstanceFromCompiledRules back to summary

getInstanceFromCompiledRules	back to summary
pack-priv static RuleBasedBreakIterator getInstanceFromCompiledRules(ByteBuffer bytes, boolean phraseBreaking) throws IOException This factory method doesn't have an access modifier; it is only accessible in the same package. Create a break iterator from a precompiled set of break rules. Creating a break iterator from the binary rules is much faster than creating one from source rules. The binary rules are generated by the RuleBasedBreakIterator.compileRules() function. Binary break iterator rules are not guaranteed to be compatible between different versions of ICU. Parameters bytes:ByteBuffer a buffer supplying the compiled binary rules. phraseBreaking:boolean a flag indicating if phrase breaking is required. Exceptions IOException: if there is an error while reading the rules from the buffer. ICU Status Internal ICU component. See Also `compileRules(String, OutputStream)`

pack-priv static RuleBasedBreakIterator getInstanceFromCompiledRules(ByteBuffer bytes, boolean phraseBreaking) throws IOException

This factory method doesn't have an access modifier; it is only accessible in the same package. Create a break iterator from a precompiled set of break rules. Creating a break iterator from the binary rules is much faster than creating one from source rules. The binary rules are generated by the RuleBasedBreakIterator.compileRules() function. Binary break iterator rules are not guaranteed to be compatible between different versions of ICU.

Parameters

bytes:ByteBuffer: a buffer supplying the compiled binary rules.
phraseBreaking:boolean: a flag indicating if phrase breaking is required.

Exceptions

IOException:: if there is an error while reading the rules from the buffer.

ICU Status

Internal ICU component.

See Also

compileRules(String, OutputStream)

getInstanceFromCompiledRules back to summary

getInstanceFromCompiledRules	back to summary
public static RuleBasedBreakIterator getInstanceFromCompiledRules(ByteBuffer bytes) throws IOException Deprecated This API is ICU internal only. Create a break iterator from a precompiled set of break rules. Creating a break iterator from the binary rules is much faster than creating one from source rules. The binary rules are generated by the RuleBasedBreakIterator.compileRules() function. Binary break iterator rules are not guaranteed to be compatible between different versions of ICU. Parameters bytes:ByteBuffer a buffer supplying the compiled binary rules. Annotations @Deprecated Exceptions IOException: if there is an error while reading the rules from the buffer. ICU Status Internal ICU component. See Also `compileRules(String, OutputStream)`

public static RuleBasedBreakIterator getInstanceFromCompiledRules(ByteBuffer bytes) throws IOException

Deprecated

This API is ICU internal only.

Parameters

bytes:ByteBuffer: a buffer supplying the compiled binary rules.

Annotations

@Deprecated

Exceptions

IOException:: if there is an error while reading the rules from the buffer.

ICU Status

Internal ICU component.

See Also

compileRules(String, OutputStream)

getLanguageBreakEngine	back to summary
private LanguageBreakEngine getLanguageBreakEngine(int c)

getRuleStatus back to summary

getRuleStatus	back to summary
public int getRuleStatus() Overrides com.ibm.icu.text.BreakIterator.getRuleStatus. Return the status tag from the break rule that determined the boundary at the current iteration position. The values appear in the rule source within brackets, {123}, for example. For rules that do not specify a status, a default value of 0 is returned. If more than one rule applies, the numerically largest of the possible status values is returned. Of the standard types of ICU break iterators, only the word and line break iterator provides status values. The values are defined in class RuleBasedBreakIterator, and allow distinguishing between words that contain alphabetic letters, "words" that appear to be numbers, punctuation and spaces, words containing ideographic characters, and more. Call `getRuleStatus` after obtaining a boundary position from `next()`, `previous()`, or any other break iterator functions that returns a boundary position. Note that `getRuleStatus()` returns the value corresponding to `current()` index even after `next()` has returned DONE. Returns:int the status from the break rule that determined the boundary at the current iteration position. Annotations @Override ICU Status Stable since ICU 60.

public int getRuleStatus()

Overrides com.ibm.icu.text.BreakIterator.getRuleStatus.

Return the status tag from the break rule that determined the boundary at the current iteration position. The values appear in the rule source within brackets, {123}, for example. For rules that do not specify a status, a default value of 0 is returned. If more than one rule applies, the numerically largest of the possible status values is returned.

Of the standard types of ICU break iterators, only the word and line break iterator provides status values. The values are defined in class RuleBasedBreakIterator, and allow distinguishing between words that contain alphabetic letters, "words" that appear to be numbers, punctuation and spaces, words containing ideographic characters, and more. Call getRuleStatus after obtaining a boundary position from next(), previous(), or any other break iterator functions that returns a boundary position.

Note that getRuleStatus() returns the value corresponding to current() index even after next() has returned DONE.

Returns:int

the status from the break rule that determined the boundary at the current iteration position.

Annotations

@Override

ICU Status

Stable since ICU 60.

getRuleStatusVec	back to summary
public int getRuleStatusVec(int[] fillInArray) Overrides com.ibm.icu.text.BreakIterator.getRuleStatusVec. Get the status (tag) values from the break rule(s) that determined the boundary at the current iteration position. The values appear in the rule source within brackets, {123}, for example. The default status value for rules that do not explicitly provide one is zero. The status values used by the standard ICU break rules are defined as public constants in class RuleBasedBreakIterator. If the size of the output array is insufficient to hold the data, the output will be truncated to the available length. No exception will be thrown. Parameters fillInArray:int[] an array to be filled in with the status values. Returns:int The number of rule status values from the rules that determined the boundary at the current iteration position. In the event that the array is too small, the return value is the total number of status values that were available, not the reduced number that were actually returned. Annotations @Override ICU Status Stable since ICU 60.

getRuleStatusVec

back to summary

public int getRuleStatusVec(int[] fillInArray)

Overrides com.ibm.icu.text.BreakIterator.getRuleStatusVec.

Get the status (tag) values from the break rule(s) that determined the boundary at the current iteration position. The values appear in the rule source within brackets, {123}, for example. The default status value for rules that do not explicitly provide one is zero.

The status values used by the standard ICU break rules are defined as public constants in class RuleBasedBreakIterator.

If the size of the output array is insufficient to hold the data, the output will be truncated to the available length. No exception will be thrown.

Parameters

fillInArray:int[]: an array to be filled in with the status values.

Returns:int

The number of rule status values from the rules that determined the boundary at the current iteration position. In the event that the array is too small, the return value is the total number of status values that were available, not the reduced number that were actually returned.

Annotations

@Override

ICU Status

Stable since ICU 60.

getText	back to summary
public CharacterIterator getText() Implements abstract com.ibm.icu.text.BreakIterator.getText. Returns a CharacterIterator over the text being analyzed. Caution The state of the returned CharacterIterator must not be modified in any way while the BreakIterator is still in use. Doing so will lead to undefined behavior of the BreakIterator. Clone the returned CharacterIterator first and work with that. The returned CharacterIterator is a reference to the actual iterator being used by the BreakIterator. No guarantees are made about the current position of this iterator when it is returned; it may differ from the BreakIterators current position. If you need to move that position to examine the text, clone this function's return value first. Returns:CharacterIterator An iterator over the text being analyzed. Annotations @Override ICU Status Stable since ICU 2.0.

getText

back to summary

public CharacterIterator getText()

Implements abstract com.ibm.icu.text.BreakIterator.getText.

Returns a CharacterIterator over the text being analyzed.

Caution

The state of the returned CharacterIterator must not be modified in any way while the BreakIterator is still in use. Doing so will lead to undefined behavior of the BreakIterator. Clone the returned CharacterIterator first and work with that.

The returned CharacterIterator is a reference to the actual iterator being used by the BreakIterator. No guarantees are made about the current position of this iterator when it is returned; it may differ from the BreakIterators current position. If you need to move that position to examine the text, clone this function's return value first.

Returns:CharacterIterator

An iterator over the text being analyzed.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

handleNext	back to summary
private int handleNext() The State Machine Engine for moving forward is here. This function is the heart of the RBBI run time engine. Input fPosition, the position in the text to begin from. Output fPosition: the boundary following the starting position. fDictionaryCharCount the number of dictionary characters encountered. If > 0, the segment will be further subdivided fRuleStatusIndex Info from the state table indicating which rules caused the boundary. Returns:int the new iterator position A note on supplementary characters and the position of underlying Java CharacterIterator: Normally, a character iterator is positioned at the char most recently returned by next(). Within this function, when a supplementary char is being processed, the char iterator is left sitting on the trail surrogate, in the middle of the code point. This is different from everywhere else, where an iterator always points at the lead surrogate of a supplementary.

handleNext

back to summary

private int handleNext()

The State Machine Engine for moving forward is here. This function is the heart of the RBBI run time engine. Input fPosition, the position in the text to begin from. Output fPosition: the boundary following the starting position. fDictionaryCharCount the number of dictionary characters encountered. If > 0, the segment will be further subdivided fRuleStatusIndex Info from the state table indicating which rules caused the boundary.

Returns:int: the new iterator position A note on supplementary characters and the position of underlying Java CharacterIterator: Normally, a character iterator is positioned at the char most recently returned by next(). Within this function, when a supplementary char is being processed, the char iterator is left sitting on the trail surrogate, in the middle of the code point. This is different from everywhere else, where an iterator always points at the lead surrogate of a supplementary.

handleSafePrevious	back to summary
private int handleSafePrevious(int fromPosition) Iterate backwards from an arbitrary position in the input text using the Safe Reverse rules. This locates a "Safe Position" from which the forward break rules will operate correctly. A Safe Position is not necessarily a boundary itself. The logic of this function is very similar to handleNext(), above, but simpler because the safe table does not require as many options. Parameters fromPosition:int the position in the input text to begin the iteration. ICU Status Internal ICU component.

handleSafePrevious

back to summary

private int handleSafePrevious(int fromPosition)

Iterate backwards from an arbitrary position in the input text using the Safe Reverse rules. This locates a "Safe Position" from which the forward break rules will operate correctly. A Safe Position is not necessarily a boundary itself. The logic of this function is very similar to handleNext(), above, but simpler because the safe table does not require as many options.

Parameters

fromPosition:int: the position in the input text to begin the iteration.

ICU Status

Internal ICU component.

hashCode	back to summary
public int hashCode() Overrides java.lang.Object.hashCode. Compute a hashcode for this BreakIterator Returns:int A hash code Annotations @Override ICU Status Stable since ICU 2.0.

hashCode

back to summary

public int hashCode()

Overrides java.lang.Object.hashCode.

Compute a hashcode for this BreakIterator

Returns:int

A hash code

Annotations

@Override

ICU Status

Stable since ICU 2.0.

isBoundary	back to summary
public boolean isBoundary(int offset) Overrides com.ibm.icu.text.BreakIterator.isBoundary. Returns true if the specified position is a boundary position. As a side effect, leaves the iterator pointing to the first boundary position at or after "offset". Parameters offset:int the offset to check. Returns:boolean True if "offset" is a boundary position. Annotations @Override ICU Status Stable since ICU 2.0.

isBoundary

back to summary

public boolean isBoundary(int offset)

Overrides com.ibm.icu.text.BreakIterator.isBoundary.

Returns true if the specified position is a boundary position. As a side effect, leaves the iterator pointing to the first boundary position at or after "offset".

Parameters

offset:int: the offset to check.

Returns:boolean

True if "offset" is a boundary position.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

last	back to summary
public int last() Implements abstract com.ibm.icu.text.BreakIterator.last. Sets the current iteration position to the end of the text. (i.e., the CharacterIterator's ending offset). Returns:int The text's past-the-end offset. Annotations @Override ICU Status Stable since ICU 2.0.

last

back to summary

public int last()

Implements abstract com.ibm.icu.text.BreakIterator.last.

Sets the current iteration position to the end of the text. (i.e., the CharacterIterator's ending offset).

Returns:int

The text's past-the-end offset.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

next	back to summary
public int next(int n) Implements abstract com.ibm.icu.text.BreakIterator.next. Advances the iterator either forward or backward the specified number of steps. Negative values move backward, and positive values move forward. This is equivalent to repeatedly calling next() or previous(). Parameters n:int The number of steps to move. The sign indicates the direction (negative is backwards, and positive is forwards). Returns:int The character offset of the boundary position n boundaries away from the current one. Annotations @Override ICU Status Stable since ICU 2.0.

back to summary

public int next(int n)

Implements abstract com.ibm.icu.text.BreakIterator.next.

Advances the iterator either forward or backward the specified number of steps. Negative values move backward, and positive values move forward. This is equivalent to repeatedly calling next() or previous().

Parameters

n:int: The number of steps to move. The sign indicates the direction (negative is backwards, and positive is forwards).

Returns:int

The character offset of the boundary position n boundaries away from the current one.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

next	back to summary
public int next() Implements abstract com.ibm.icu.text.BreakIterator.next. Advances the iterator to the next boundary position. Returns:int The position of the first boundary after this one. Annotations @Override ICU Status Stable since ICU 2.0.

back to summary

public int next()

Implements abstract com.ibm.icu.text.BreakIterator.next.

Advances the iterator to the next boundary position.

Returns:int

The position of the first boundary after this one.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

preceding

back to summary

public int preceding(int offset)

Overrides com.ibm.icu.text.BreakIterator.preceding.

Sets the iterator to refer to the last boundary position before the specified position.

Parameters

offset:int: The position to begin searching for a break from.

Returns:int

The position of the last boundary before the starting position.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

back to summary

public int previous()

Implements abstract com.ibm.icu.text.BreakIterator.previous.

Moves the iterator backwards, to the boundary preceding the current one.

Returns:int

The position of the boundary position immediately preceding the starting position.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

setText

back to summary

public void setText(CharacterIterator newText)

Implements abstract com.ibm.icu.text.BreakIterator.setText.

Set the iterator to analyze a new piece of text. This function resets the current iteration position to the beginning of the text. (The old iterator is dropped.)

Caution

The supplied CharacterIterator is used directly by the BreakIterator, and must not be altered in any way by code outside of the BreakIterator. Doing so will lead to undefined behavior of the BreakIterator.

Parameters

newText:CharacterIterator: An iterator over the text to analyze.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

toString

back to summary

public String toString()

Overrides java.lang.Object.toString.

Returns the description (rules) used to create this iterator. (In ICU4C, the same function is RuleBasedBreakIterator::getRules())

Returns:String

Doc from java.lang.Object.toString.

a string representation of the object.

Annotations

@Override

ICU Status

Stable since ICU 2.0.

com.ibm.icu.text back to summary

pack-priv Class RuleBasedBreakIterator.BreakCache

Class Inheritance

extends Object

java.lang.Object
com.ibm.icu.text.RuleBasedBreakIterator.BreakCache

Field Summary

Modifier and Type	Field and Description
pack-priv static final int	CACHE_SIZE
pack-priv int[]	fBoundaries
pack-priv int	fBufIdx
pack-priv int	fEndBufIdx
pack-priv DictionaryBreakEngine.DequeI	fSideBuffer
pack-priv int	fStartBufIdx
pack-priv short[]	fStatuses
pack-priv int	fTextIdx
pack-priv static final boolean	RetainCachePosition
pack-priv static final boolean	UpdateCachePosition

Constructor Summary

Access	Constructor and Description
pack-priv	BreakCache()
pack-priv	BreakCache(RuleBasedBreakIterator.BreakCache src) copy constructor, used from RuleBasedBreakIterator.clone().

Method Summary

Modifier and Type	Method and Description
pack-priv void	addFollowing(int position, int ruleStatusIdx, boolean update) Add the boundary following the current position.
pack-priv boolean	addPreceding(int position, int ruleStatusIdx, boolean update) Add the boundary preceding the current position.
pack-priv int	current() Update the state of the public BreakIterator (fBI) to reflect the current state of the break iterator cache (this).
pack-priv void	dumpCache()
pack-priv void	following(int startPos)
private final int	modChunkSize(int index)
pack-priv void	next()
pack-priv boolean	populateFollowing() Add boundary(s) to the cache following the current last boundary.
pack-priv boolean	populateNear(int position) Add boundaries to the cache near the specified position.
pack-priv boolean	populatePreceding() Add one or more boundaries to the cache preceding the first currently cached boundary.
pack-priv void	preceding(int startPos)
pack-priv void	previous()
pack-priv void	reset(int pos, int ruleStatus)
pack-priv void	reset()
pack-priv boolean	seek(int pos) Set the cache position to the specified position, or, if the position falls between to cached boundaries, to the preceding boundary.

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

Field Detail

CACHE_SIZE	back to summary
pack-priv static final int CACHE_SIZE

fBoundaries	back to summary
pack-priv int[] fBoundaries

fBufIdx	back to summary
pack-priv int fBufIdx

fEndBufIdx	back to summary
pack-priv int fEndBufIdx

fSideBuffer	back to summary
pack-priv DictionaryBreakEngine.DequeI fSideBuffer

fStartBufIdx	back to summary
pack-priv int fStartBufIdx

fStatuses	back to summary
pack-priv short[] fStatuses

fTextIdx	back to summary
pack-priv int fTextIdx

RetainCachePosition	back to summary
pack-priv static final boolean RetainCachePosition

UpdateCachePosition	back to summary
pack-priv static final boolean UpdateCachePosition

Constructor Detail

BreakCache	back to summary
pack-priv BreakCache()

BreakCache	back to summary
pack-priv BreakCache(RuleBasedBreakIterator.BreakCache src) copy constructor, used from RuleBasedBreakIterator.clone().

Method Detail

addFollowing	back to summary
pack-priv void addFollowing(int position, int ruleStatusIdx, boolean update) Add the boundary following the current position. The current position can be left as it was, or changed to the newly added boundary, as specified by the update parameter.

addPreceding	back to summary
pack-priv boolean addPreceding(int position, int ruleStatusIdx, boolean update) Add the boundary preceding the current position. The current position can be left as it was, or changed to the newly added boundary, as specified by the update parameter.

current	back to summary
pack-priv int current() Update the state of the public BreakIterator (fBI) to reflect the current state of the break iterator cache (this).

dumpCache	back to summary
pack-priv void dumpCache()

following	back to summary
pack-priv void following(int startPos)

modChunkSize	back to summary
private final int modChunkSize(int index)

next	back to summary
pack-priv void next()

populateFollowing	back to summary
pack-priv boolean populateFollowing() Add boundary(s) to the cache following the current last boundary. Return false if at the end of the text, and no more boundaries can be added. Leave iteration position at the first newly added boundary, or unchanged if no boundary was added.

populateNear

back to summary

pack-priv boolean populateNear(int position)

Add boundaries to the cache near the specified position. The given position need not be a boundary itself. The input position must be within the range of the text, and on a code point boundary. If the requested position is a break boundary, leave the iteration position on it. If the requested position is not a boundary, leave the iteration position on the preceding boundary and include both the the preceding and following boundaries in the cache. Additional boundaries, either preceding or following, may be added to the cache as a side effect. Return false if the operation failed.

populatePreceding	back to summary
pack-priv boolean populatePreceding() Add one or more boundaries to the cache preceding the first currently cached boundary. Leave the iteration position on the first added boundary. Return false if no boundaries could be added (if at the start of the text.)

preceding	back to summary
pack-priv void preceding(int startPos)

previous	back to summary
pack-priv void previous()

reset	back to summary
pack-priv void reset(int pos, int ruleStatus)

reset	back to summary
pack-priv void reset()

seek

back to summary

pack-priv boolean seek(int pos)

Set the cache position to the specified position, or, if the position falls between to cached boundaries, to the preceding boundary. Fails if the requested position is outside of the range of boundaries currently held by the cache. The startPosition must be on a code point boundary. Return true if successful, false if the specified position is after the last cached boundary or before the first.

com.ibm.icu.text back to summary

pack-priv Class RuleBasedBreakIterator.DictionaryCache

Class Inheritance

extends Object

java.lang.Object
com.ibm.icu.text.RuleBasedBreakIterator.DictionaryCache

DictionaryCache stores the boundaries obtained from a run of dictionary characters. Dictionary boundaries are moved first to this cache, then from here to the main BreakCache, where they may inter-leave with non-dictionary boundaries. The public BreakIterator API always fetches directly from the main BreakCache, not from here. In common situations, the number of boundaries in a single dictionary run should be quite small, it will be terminated by punctuation, spaces, or any other non-dictionary characters. The main BreakCache may end up with boundaries from multiple dictionary based runs. The boundaries are stored in a simple ArrayList (vector), with the assumption that they will be accessed sequentially.

Field Summary

Modifier and Type	Field and Description
pack-priv int	fBoundary
pack-priv DictionaryBreakEngine.DequeI	fBreaks
pack-priv int	fFirstRuleStatusIndex
pack-priv int	fLimit
pack-priv int	fOtherRuleStatusIndex
pack-priv int	fPositionInCache
pack-priv int	fStart
pack-priv int	fStatusIndex

Constructor Summary

Access	Constructor and Description
pack-priv	DictionaryCache()
pack-priv	DictionaryCache(RuleBasedBreakIterator.DictionaryCache the source object to be copied. src) copy constructor.

Method Summary

Modifier and Type	Method and Description
pack-priv boolean	following(int fromPos)
pack-priv void	populateDictionary(int The start position of a range of text startPos, int The end position of a range of text endPos, int The rule status index that applies to the break at startPos firstRuleStatus, int The rule status index that applies to boundaries other than startPos otherRuleStatus) Populate the cache with the dictionary based boundaries within a region of text.
pack-priv boolean	preceding(int fromPos)
pack-priv void	reset()

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

Field Detail

fBoundary	back to summary
pack-priv int fBoundary

fBreaks	back to summary
pack-priv DictionaryBreakEngine.DequeI fBreaks

fFirstRuleStatusIndex	back to summary
pack-priv int fFirstRuleStatusIndex

fLimit	back to summary
pack-priv int fLimit

fOtherRuleStatusIndex	back to summary
pack-priv int fOtherRuleStatusIndex

fPositionInCache	back to summary
pack-priv int fPositionInCache

fStart	back to summary
pack-priv int fStart

fStatusIndex	back to summary
pack-priv int fStatusIndex

Constructor Detail

DictionaryCache	back to summary
pack-priv DictionaryCache()

DictionaryCache

back to summary

pack-priv DictionaryCache(RuleBasedBreakIterator.DictionaryCache src)

copy constructor. Used by RuleBasedBreakIterator.clone().

Parameters

src:RuleBasedBreakIterator.DictionaryCache: the source object to be copied.

Method Detail

following	back to summary
pack-priv boolean following(int fromPos)

populateDictionary

back to summary

pack-priv void populateDictionary(int startPos, int endPos, int firstRuleStatus, int otherRuleStatus)

Populate the cache with the dictionary based boundaries within a region of text.

Parameters

startPos:int: The start position of a range of text
endPos:int: The end position of a range of text
firstRuleStatus:int: The rule status index that applies to the break at startPos
otherRuleStatus:int: The rule status index that applies to boundaries other than startPos

ICU Status

Internal ICU component.

preceding	back to summary
pack-priv boolean preceding(int fromPos)

reset	back to summary
pack-priv void reset()

public Class RuleBasedBreakIterator

Nested and Inner Type Summary

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail

pack-priv Class RuleBasedBreakIterator.BreakCache

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail

pack-priv Class RuleBasedBreakIterator.DictionaryCache

Field Summary

Constructor Summary

Method Summary

Field Detail

Constructor Detail

Method Detail