Top Description Fields Constructors Methods
org.apache.commons.text.similarity

pack-priv final Class RegexTokenizer

extends Object
implements CharSequenceTokenizer<CharSequence>
Class Inheritance
All Implemented Interfaces
org.apache.commons.text.similarity.CharSequenceTokenizer, org.apache.commons.text.similarity.Tokenizer, java.util.function.Function
Imports
java.util.ArrayList, .List, java.util.regex.Matcher, .Pattern, org.apache.commons.lang3.ArrayUtils, .StringUtils, .Validate
A simple word Tokenizer that utilizes a regex to find words. It applies a regex (\w)+ over the input text to extract words from a given character sequence.

Instances of this class are immutable and are safe for use by multiple concurrent threads.

Since
1.0

Field Summary

Modifier and TypeField and Description
pack-priv static final RegexTokenizer
INSTANCE

The singleton instance.

private static final Pattern
PATTERN

The whitespace pattern.

Constructor Summary

AccessConstructor and Description
pack-priv
RegexTokenizer()

Method Summary

Modifier and TypeMethod and Description
public CharSequence[]
apply(final CharSequence text)

Implements java.util.function.Function.apply.

Applies this function to the given argument.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

INSTANCEback to summary
pack-priv static final RegexTokenizer INSTANCE

The singleton instance.

PATTERNback to summary
private static final Pattern PATTERN

The whitespace pattern.

Constructor Detail

RegexTokenizerback to summary
pack-priv RegexTokenizer()

Method Detail

applyback to summary
public CharSequence[] apply(final CharSequence text)

Implements java.util.function.Function.apply.

Doc from java.util.function.Function.apply.

Applies this function to the given argument.

Annotations
@Override
Exceptions
IllegalArgumentException:
if the input text is blank.