Top Description Fields Constructors Methods
com.ibm.icu.impl

public Class UnicodeRegex

extends Object
implements Cloneable, Freezable<UnicodeRegex>, StringTransform
Class Inheritance
All Implemented Interfaces
com.ibm.icu.text.StringTransform, com.ibm.icu.text.Transform, com.ibm.icu.util.Freezable, java.lang.Cloneable
Imports
java.io.BufferedReader, .FileInputStream, .IOException, .InputStream, .InputStreamReader, .UnsupportedEncodingException, java.text.ParsePosition, java.util.Arrays, .Comparator, .LinkedHashSet, .List, .Map, .Map.Entry, .Set, .TreeMap, java.util.regex.Pattern, com.ibm.icu.text.StringTransform, .SymbolTable, .UnicodeSet, com.ibm.icu.util.Freezable

Contains utilities to supplement the JDK Regex, since it doesn't handle Unicode well.

Todo

Move to com.ibm.icu.dev.somewhere. 2015-sep-03: This is used there, and also in CLDR and in UnicodeTools.

Author
markdavis

Field Summary

Modifier and TypeField and Description
private String
private String
private String
private Comparator<Object>
private static final UnicodeRegex
private static final Pattern
private SymbolTable

Constructor Summary

AccessConstructor and Description
public

Method Summary

Modifier and TypeMethod and Description
public static List<String>

Returns:

filled list
appendLines
(List<String>
The result of the appended lines.
result
,
String
The file to have an input stream.
file
,
String
if null, then UTF-8
encoding
)

Utility for loading lines from a file.

public static List<String>

Returns:

filled list
appendLines
(List<String>
The result of the appended lines.
result
,
InputStream
The input stream.
inputStream
,
String
if null, then UTF-8
encoding
)

Utility for loading lines from a UTF8 file.

public UnicodeRegex
cloneAsThawed()

Implements com.ibm.icu.util.Freezable.cloneAsThawed.

Provides for the clone operation.
public static Pattern

Returns:

Pattern
compile
(String
Raw regex pattern, as in fix(...).
regex
)

Compile a regex string, after processing by fix(...).

public static Pattern

Returns:

Pattern
compile
(String
Raw regex pattern, as in fix(...).
regex
,
int options)

Compile a regex string, after processing by fix(...).

public String

Returns:

Pattern
compileBnf
(String
Series of BNF lines.
bnfLines
)

Compile a composed string from a set of BNF lines; see the List version for more information.

public String

Returns:

Pattern
compileBnf
(List<String>
Series of lines that represent a BNF expression. The lines contain a series of statements that of the form x=y;. A statement can take multiple lines, but there can't be multiple statements on a line. A hash quotes to the end of the line.
lines
)

Compile a composed string from a set of BNF lines, such as for composing a regex expression.

public static String

Returns:

processed regex pattern, as in process()
fix
(String
as in process()
regex
)

Convenience static function, using standard parameters.

public UnicodeRegex
freeze()

Implements com.ibm.icu.util.Freezable.freeze.

Freezes the object.
public String
public String
public String
public SymbolTable
getSymbolTable()

Set the symbol table for internal processing

private Map<String, String>
public boolean
isFrozen()

Implements com.ibm.icu.util.Freezable.isFrozen.

Determines whether the object has been frozen or not.
private int
processSet(String regex, int i, StringBuilder result, UnicodeSet temp, ParsePosition pos)

public void
setBnfCommentString(String bnfCommentString)

public void
setBnfLineSeparator(String bnfLineSeparator)

public void
setBnfVariableInfix(String bnfVariableInfix)

public UnicodeRegex
setSymbolTable(SymbolTable symbolTable)

Get the symbol table for internal processing

public String

Returns:

A processed Java regex pattern, suitable for input to Pattern.compile().
transform
(String
A modified Java regex pattern, as in the input to Pattern.compile(), except that all "character classes" are processed as if they were UnicodeSet patterns. Example: "abc[:bc=N:]. See UnicodeSet for the differences in syntax.
regex
)

Implements com.ibm.icu.text.StringTransform.transform.

Adds full Unicode property support, with the latest version of Unicode, to Java Regex, bringing it up to Level 1 (see http://www.unicode.org/reports/tr18/).
Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

bnfCommentStringback to summary
private String bnfCommentString
bnfLineSeparatorback to summary
private String bnfLineSeparator
bnfVariableInfixback to summary
private String bnfVariableInfix
LongestFirstback to summary
private Comparator<Object> LongestFirst
STANDARDback to summary
private static final UnicodeRegex STANDARD
SUPP_ESCAPEback to summary
private static final Pattern SUPP_ESCAPE
symbolTableback to summary
private SymbolTable symbolTable

Constructor Detail

UnicodeRegexback to summary
public UnicodeRegex()

Method Detail

appendLinesback to summary
public static List<String> appendLines(List<String> result, String file, String encoding) throws IOException

Utility for loading lines from a file.

Parameters
result:List<String>

The result of the appended lines.

file:String

The file to have an input stream.

encoding:String

if null, then UTF-8

Returns:List<String>

filled list

Exceptions
IOException:
If there were problems opening the file for input stream.
appendLinesback to summary
public static List<String> appendLines(List<String> result, InputStream inputStream, String encoding) throws UnsupportedEncodingException, IOException

Utility for loading lines from a UTF8 file.

Parameters
result:List<String>

The result of the appended lines.

inputStream:InputStream

The input stream.

encoding:String

if null, then UTF-8

Returns:List<String>

filled list

Exceptions
IOException:
If there were problems opening the input stream for reading.
cloneAsThawedback to summary
public UnicodeRegex cloneAsThawed()

Implements com.ibm.icu.util.Freezable.cloneAsThawed.

Doc from com.ibm.icu.util.Freezable.cloneAsThawed.

Provides for the clone operation. Any clone is initially unfrozen.

Annotations
@Override
compileback to summary
public static Pattern compile(String regex)

Compile a regex string, after processing by fix(...).

Parameters
regex:String

Raw regex pattern, as in fix(...).

Returns:Pattern

Pattern

compileback to summary
public static Pattern compile(String regex, int options)

Compile a regex string, after processing by fix(...).

Parameters
regex:String

Raw regex pattern, as in fix(...).

Returns:Pattern

Pattern

compileBnfback to summary
public String compileBnf(String bnfLines)

Compile a composed string from a set of BNF lines; see the List version for more information.

Parameters
bnfLines:String

Series of BNF lines.

Returns:String

Pattern

compileBnfback to summary
public String compileBnf(List<String> lines)

Compile a composed string from a set of BNF lines, such as for composing a regex expression. The lines can be in any order, but there must not be any cycles. The result can be used as input for fix().

Example:

uri = (?: (scheme) \\:)? (host) (?: \\? (query))? (?: \\u0023 (fragment))?;
scheme = reserved+;
host = // reserved+;
query = [\\=reserved]+;
fragment = reserved+;
reserved = [[:ascii:][:alphabetic:]];

Caveats: at this point the parsing is simple; for example, # cannot be quoted (use \\u0023); you can set it to null to disable. The equality sign and a few others can be reset with setBnfX().

Parameters
lines:List<String>

Series of lines that represent a BNF expression. The lines contain a series of statements that of the form x=y;. A statement can take multiple lines, but there can't be multiple statements on a line. A hash quotes to the end of the line.

Returns:String

Pattern

fixback to summary
public static String fix(String regex)

Convenience static function, using standard parameters.

Parameters
regex:String

as in process()

Returns:String

processed regex pattern, as in process()

freezeback to summary
public UnicodeRegex freeze()

Implements com.ibm.icu.util.Freezable.freeze.

Doc from com.ibm.icu.util.Freezable.freeze.

Freezes the object.

Returns:UnicodeRegex

the object itself.

Annotations
@Override
getBnfCommentStringback to summary
public String getBnfCommentString()
getBnfLineSeparatorback to summary
public String getBnfLineSeparator()
getBnfVariableInfixback to summary
public String getBnfVariableInfix()
getSymbolTableback to summary
public SymbolTable getSymbolTable()

Set the symbol table for internal processing

ICU Status
Internal ICU component.
getVariablesback to summary
private Map<String, String> getVariables(List<String> lines)
isFrozenback to summary
public boolean isFrozen()

Implements com.ibm.icu.util.Freezable.isFrozen.

Doc from com.ibm.icu.util.Freezable.isFrozen.

Determines whether the object has been frozen or not.

Annotations
@Override
processSetback to summary
private int processSet(String regex, int i, StringBuilder result, UnicodeSet temp, ParsePosition pos)
setBnfCommentStringback to summary
public void setBnfCommentString(String bnfCommentString)
setBnfLineSeparatorback to summary
public void setBnfLineSeparator(String bnfLineSeparator)
setBnfVariableInfixback to summary
public void setBnfVariableInfix(String bnfVariableInfix)
setSymbolTableback to summary
public UnicodeRegex setSymbolTable(SymbolTable symbolTable)

Get the symbol table for internal processing

ICU Status
Internal ICU component.
transformback to summary
public String transform(String regex)

Implements com.ibm.icu.text.StringTransform.transform.

Adds full Unicode property support, with the latest version of Unicode, to Java Regex, bringing it up to Level 1 (see http://www.unicode.org/reports/tr18/). It does this by preprocessing the regex pattern string and interpreting the character classes (\p{...}, \P{...}, [...]) according to their syntax and meaning in UnicodeSet. With this utility, Java regex expressions can be updated to work with the latest version of Unicode, and with all Unicode properties. Note that the UnicodeSet syntax has not yet, however, been updated to be completely consistent with Java regex, so be careful of the differences.

Not thread-safe; create a separate copy for different threads.

In the future, we may extend this to support other regex packages.

Parameters
regex:String

A modified Java regex pattern, as in the input to Pattern.compile(), except that all "character classes" are processed as if they were UnicodeSet patterns. Example: "abc[:bc=N:]. See UnicodeSet for the differences in syntax.

Returns:String

A processed Java regex pattern, suitable for input to Pattern.compile().

Annotations
@Override