Top Fields Constructors Methods
com.ibm.icu.impl.breakiter

public Class MlBreakEngine

Additional top-level class in compilation unit: ModelIndex.

extends Object
Class Inheritance
Static Imports
com.ibm.icu.impl.CharacterIteration.DONE32, .CharacterIteration.current32, .CharacterIteration.next32, .CharacterIteration.previous32

Field Summary

Modifier and TypeField and Description
private UnicodeSet
fClosePunctuationSet

private UnicodeSet
fDigitOrOpenPunctuationOrAlphabetSet

private List<HashMap<String, Integer>>
fModel

private int
fNegativeSum

private static final int
MAX_FEATURE

Constructor Summary

AccessConstructor and Description
public
MlBreakEngine(UnicodeSet
An unicode set with the digit and open punctuation and alphabet.
digitOrOpenPunctuationOrAlphabetSet, UnicodeSet
An unicode set with the close punctuation.
closePunctuationSet)

Constructor for Chinese and Japanese phrase breaking.

Method Summary

Modifier and TypeMethod and Description
public int

Returns:

The number of breakpoints
divideUpRange(CharacterIterator
An input text.
inText, int
The start index of the input text.
startPos, int
The end index of the input text.
endPos, CharacterIterator
A input string normalized from inText from startPos to endPos
inString, int
The number of code points of inString
codePointLength, int[]
A map that transforms inString's code point index to code unit index.
charPositions, DictionaryBreakEngine.DequeI
A list to store the breakpoint.
foundBreaks)

Divide up a range of characters handled by this break engine.

private void
evaluateBreakpoint(String
An input string to be segmented.
inputStr, int[]
A code unit index list of the inputStr.
indexList, int
The start index of the indexList.
startIdx, int
The current code unit boundary of the indexList.
numCodeUnits, ArrayList<Integer>
A list including the index of the breakpoint.
boundary)

Evaluate whether the breakpointIdx is a potential breakpoint.

private int

Returns:

The number of the code units of the first six characters in inString.
initIndexList(CharacterIterator
An input string to be segmented.
inString, int[]
A code unit index list of the inString.
indexList, int
The number of code points of the input string
codePointLength)

Initialize the index list from the input string.

private void
initKeyValue(UResourceBundle
A RedouceBundle corresponding to the model file.
rb, String
The kay name in the model file.
keyName, String
The value name in the model file.
valueName, HashMap<String, Integer>
A HashMap to store the pairs of the feature and its score.
map)

In the machine learning's model file, specify the name of the key and value to load the corresponding feature and its score.

private void
loadMLModel()

Load the machine learning's model file.

private String
transform(CharacterIterator inString)

Transform a CharacterIterator into a String.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

fClosePunctuationSetback to summary
private UnicodeSet fClosePunctuationSet
fDigitOrOpenPunctuationOrAlphabetSetback to summary
private UnicodeSet fDigitOrOpenPunctuationOrAlphabetSet
fModelback to summary
private List<HashMap<String, Integer>> fModel
fNegativeSumback to summary
private int fNegativeSum
MAX_FEATUREback to summary
private static final int MAX_FEATURE

Constructor Detail

MlBreakEngineback to summary
public MlBreakEngine(UnicodeSet digitOrOpenPunctuationOrAlphabetSet, UnicodeSet closePunctuationSet)

Constructor for Chinese and Japanese phrase breaking.

Parameters
digitOrOpenPunctuationOrAlphabetSet:UnicodeSet

An unicode set with the digit and open punctuation and alphabet.

closePunctuationSet:UnicodeSet

An unicode set with the close punctuation.

Method Detail

divideUpRangeback to summary
public int divideUpRange(CharacterIterator inText, int startPos, int endPos, CharacterIterator inString, int codePointLength, int[] charPositions, DictionaryBreakEngine.DequeI foundBreaks)

Divide up a range of characters handled by this break engine.

Parameters
inText:CharacterIterator

An input text.

startPos:int

The start index of the input text.

endPos:int

The end index of the input text.

inString:CharacterIterator

A input string normalized from inText from startPos to endPos

codePointLength:int

The number of code points of inString

charPositions:int[]

A map that transforms inString's code point index to code unit index.

foundBreaks:DictionaryBreakEngine.DequeI

A list to store the breakpoint.

Returns:int

The number of breakpoints

evaluateBreakpointback to summary
private void evaluateBreakpoint(String inputStr, int[] indexList, int startIdx, int numCodeUnits, ArrayList<Integer> boundary)

Evaluate whether the breakpointIdx is a potential breakpoint.

Parameters
inputStr:String

An input string to be segmented.

indexList:int[]

A code unit index list of the inputStr.

startIdx:int

The start index of the indexList.

numCodeUnits:int

The current code unit boundary of the indexList.

boundary:ArrayList<Integer>

A list including the index of the breakpoint.

initIndexListback to summary
private int initIndexList(CharacterIterator inString, int[] indexList, int codePointLength)

Initialize the index list from the input string.

Parameters
inString:CharacterIterator

An input string to be segmented.

indexList:int[]

A code unit index list of the inString.

codePointLength:int

The number of code points of the input string

Returns:int

The number of the code units of the first six characters in inString.

initKeyValueback to summary
private void initKeyValue(UResourceBundle rb, String keyName, String valueName, HashMap<String, Integer> map)

In the machine learning's model file, specify the name of the key and value to load the corresponding feature and its score.

Parameters
rb:UResourceBundle

A RedouceBundle corresponding to the model file.

keyName:String

The kay name in the model file.

valueName:String

The value name in the model file.

map:HashMap<String, Integer>

A HashMap to store the pairs of the feature and its score.

loadMLModelback to summary
private void loadMLModel()

Load the machine learning's model file.

transformback to summary
private String transform(CharacterIterator inString)

Transform a CharacterIterator into a String.