Apache PDFBox 3.0.4 · org.apache.fontbox.ttf.gsub.CompoundCharacterTokenizer

Takes in the given text having compound-glyphs to substitute, and splits it into chunks consisting of parts that should be substituted and the ones that can be processed normally.

Field Summary

Modifier and Type	Field and Description
private static final String	GLYPH_ID_SEPARATOR
private final Pattern	regexExpression

Constructor Summary

Access	Constructor and Description
public	CompoundCharacterTokenizer(Set<String> A set of strings like _79_99_, _80_99_ or _92_99_ . compoundWords) Constructor.
public	CompoundCharacterTokenizer(Pattern pattern)

Method Summary

Modifier and Type	Method and Description
private String	getRegexFromTokens(Set<String> compoundWords)
public List<String>	Returns: A list of tokens like "_66_", "_71_71_", "74_79_70_". The "_" is sometimes missing at the beginning or end, this has to be cleaned by the caller. tokenize(String A string like "_66_71_71_74_79_70_" text) Tokenize a string into tokens.
private void	validateCompoundWords(Set<String> compoundWords) Validate the compound words.

Inherited from java.lang.Object:: clone equals finalize getClass hashCode notify notifyAll toString wait wait wait

Field Detail

GLYPH_ID_SEPARATOR	back to summary
private static final String GLYPH_ID_SEPARATOR

regexExpression	back to summary
private final Pattern regexExpression

Constructor Detail

CompoundCharacterTokenizer	back to summary
public CompoundCharacterTokenizer(Set<String> compoundWords) Constructor. Calls getRegexFromTokens which returns strings like (_79_99_)\|(_80_99_)\|(_92_99_) and creates a regexp assigned to regexExpression. See the code in GlyphArraySplitterRegexImpl on how these strings were created. It is assumed the compound words are sorted in descending order of length. Parameters compoundWords:Set<String> A set of strings like _79_99_, _80_99_ or _92_99_ .

CompoundCharacterTokenizer

back to summary

public CompoundCharacterTokenizer(Set<String> compoundWords)

Constructor. Calls getRegexFromTokens which returns strings like (_79_99_)|(_80_99_)|(_92_99_) and creates a regexp assigned to regexExpression. See the code in GlyphArraySplitterRegexImpl on how these strings were created.

It is assumed the compound words are sorted in descending order of length.

Parameters

compoundWords:Set<String>: A set of strings like _79_99_, _80_99_ or _92_99_ .

CompoundCharacterTokenizer	back to summary
public CompoundCharacterTokenizer(Pattern pattern)

Method Detail

getRegexFromTokens	back to summary
private String getRegexFromTokens(Set<String> compoundWords)

tokenize	back to summary
public List<String> tokenize(String text) Tokenize a string into tokens. Parameters text:String A string like "_66_71_71_74_79_70_" Returns:List<String> A list of tokens like "_66_", "_71_71_", "74_79_70_". The "_" is sometimes missing at the beginning or end, this has to be cleaned by the caller.

tokenize

back to summary

public List<String> tokenize(String text)

Tokenize a string into tokens.