Return a steady stream of tokens.
The returned tokens are one of: LITERAL, '%', FORMAT_MODIFIER, SIMPLE_KEYWORD, COMPOSITE_KEYWORD OPTION, LEFT_PARENTHESIS, and RIGHT_PARENTHESIS.
The '\' character is used as escape. It can be used to escape '_', '%', '(' and '('.
Note that there is no EOS token returned.
| Modifier and Type | Class and Description |
|---|---|
| pack-priv static enum |
| Modifier and Type | Field and Description |
|---|---|
| pack-priv final IEscapeUtil | |
| pack-priv final IEscapeUtil | |
| pack-priv final String | |
| pack-priv final int | |
| pack-priv int | |
| pack-priv TokenStream. |
| Access | Constructor and Description |
|---|---|
| pack-priv | |
| pack-priv |
| Modifier and Type | Method and Description |
|---|---|
| private void | |
| pack-priv void | |
| private void | |
| private void | |
| private void | |
| private void | |
| pack-priv void | |
| private void | |
| pack-priv List |
| escapeUtil | back to summary |
|---|---|
| pack-priv final IEscapeUtil escapeUtil | |
| optionEscapeUtil | back to summary |
|---|---|
| pack-priv final IEscapeUtil optionEscapeUtil | |
| pattern | back to summary |
|---|---|
| pack-priv final String pattern | |
| patternLength | back to summary |
|---|---|
| pack-priv final int patternLength | |
| pointer | back to summary |
|---|---|
| pack-priv int pointer | |
| state | back to summary |
|---|---|
| pack-priv TokenStream. | |
| TokenStream | back to summary |
|---|---|
| pack-priv TokenStream(String pattern) | |
| TokenStream | back to summary |
|---|---|
| pack-priv TokenStream(String pattern, IEscapeUtil escapeUtil) | |
| addValuedToken | back to summary |
|---|---|
| private void addValuedToken(int type, StringBuffer buf, List<Token> tokenList) | |
| escape | back to summary |
|---|---|
| pack-priv void escape(String escapeChars, StringBuffer buf) | |
| handleFormatModifierState | back to summary |
|---|---|
| private void handleFormatModifierState(char c, List<Token> tokenList, StringBuffer buf) | |
| handleKeywordState | back to summary |
|---|---|
| private void handleKeywordState(char c, List<Token> tokenList, StringBuffer buf) | |
| handleLiteralState | back to summary |
|---|---|
| private void handleLiteralState(char c, List<Token> tokenList, StringBuffer buf) | |
| handleRightParenthesisState | back to summary |
|---|---|
| private void handleRightParenthesisState(char c, List<Token> tokenList, StringBuffer buf) | |
| optionEscape | back to summary |
|---|---|
| pack-priv void optionEscape(String escapeChars, StringBuffer buf) | |
| processOption | back to summary |
|---|---|
| private void processOption(char c, List<Token> tokenList, StringBuffer buf) throws ScanException | |
| tokenize | back to summary |
|---|---|
| pack-priv List | |
| Modifier and Type | Field and Description |
|---|---|
| public static final TokenStream. | |
| public static final TokenStream. | |
| public static final TokenStream. | |
| public static final TokenStream. | |
| public static final TokenStream. |
| Access | Constructor and Description |
|---|---|
| private |
| Modifier and Type | Method and Description |
|---|---|
| public static TokenStream. | |
| public static TokenStream. |
| FORMAT_MODIFIER_STATE | back to summary |
|---|---|
| public static final TokenStream. | |
| KEYWORD_STATE | back to summary |
|---|---|
| public static final TokenStream. | |
| LITERAL_STATE | back to summary |
|---|---|
| public static final TokenStream. | |
| OPTION_STATE | back to summary |
|---|---|
| public static final TokenStream. | |
| RIGHT_PARENTHESIS_STATE | back to summary |
|---|---|
| public static final TokenStream. | |
| TokenizerState | back to summary |
|---|---|
| private TokenizerState() | |
| valueOf | back to summary |
|---|---|
| public static TokenStream. | |
| values | back to summary |
|---|---|
| public static TokenStream. | |