Top Description Inners Fields Constructors Methods
org.apache.hc.core5.util

public Class Tokenizer

extends Object
Class Inheritance
Known Direct Subclasses
org.apache.hc.core5.http.message.TokenParser
Annotations
@Contract
threading:IMMUTABLE
Imports
java.util.BitSet, org.apache.hc.core5.annotation.Contract, .ThreadingBehavior

Tokenizer that can be used as a foundation for more complex parsing routines. Methods of this class are designed to produce near zero intermediate garbage and make no intermediate copies of input data.

This class is immutable and thread safe.

Since
5.1

Nested and Inner Type Summary

Modifier and TypeClass and Description
public static class

Field Summary

Modifier and TypeField and Description
public static final int
public static final char
DQUOTE

Double quote

public static final char
ESCAPE

Backward slash / escape character

public static final int
public static final Tokenizer
public static final int
public static final int

Constructor Summary

AccessConstructor and Description
public

Method Summary

Modifier and TypeMethod and Description
public void
copyContent(final CharSequence
buffer with the sequence of chars to be parsed
buf
,
final Tokenizer.Cursor
defines the bounds and current position of the buffer
cursor
,
final BitSet
set of delimiting characters. Can be null if the value is delimited by a whitespace only.
delimiters
,
final StringBuilder
destination buffer
dst
)

Transfers content into the destination buffer until a whitespace character or any of the given delimiters is encountered.

public void
copyQuotedContent(final CharSequence
buffer with the sequence of chars to be parsed
buf
,
final Tokenizer.Cursor
defines the bounds and current position of the buffer
cursor
,
final StringBuilder
destination buffer
dst
)

Transfers content enclosed with quote marks into the destination buffer.

public void
copyUnquotedContent(final CharSequence
buffer with the sequence of chars to be parsed
buf
,
final Tokenizer.Cursor
defines the bounds and current position of the buffer
cursor
,
final BitSet
set of delimiting characters. Can be null if the value is delimited by a whitespace or a quote only.
delimiters
,
final StringBuilder
destination buffer
dst
)

Transfers content into the destination buffer until a whitespace character, a quote, or any of the given delimiters is encountered.

public static BitSet
INIT_BITSET(final int... b)

public static boolean
isWhitespace(final char ch)

public String
parseContent(final CharSequence
buffer with the sequence of chars to be parsed
buf
,
final Tokenizer.Cursor
defines the bounds and current position of the buffer
cursor
,
final BitSet
set of delimiting characters. Can be null if the token is not delimited by any character.
delimiters
)

Extracts from the sequence of chars a token terminated with any of the given delimiters or a whitespace characters.

public String
parseToken(final CharSequence
buffer with the sequence of chars to be parsed
buf
,
final Tokenizer.Cursor
defines the bounds and current position of the buffer
cursor
,
final BitSet
set of delimiting characters. Can be null if the token is not delimited by any character.
delimiters
)

Extracts from the sequence of chars a token terminated with any of the given delimiters discarding semantically insignificant whitespace characters.

public String
parseValue(final CharSequence
buffer with the sequence of chars to be parsed
buf
,
final Tokenizer.Cursor
defines the bounds and current position of the buffer
cursor
,
final BitSet
set of delimiting characters. Can be null if the value is not delimited by any character.
delimiters
)

Extracts from the sequence of chars a value which can be enclosed in quote marks and terminated with any of the given delimiters discarding semantically insignificant whitespace characters.

public void
skipWhiteSpace(final CharSequence
buffer with the sequence of chars to be parsed
buf
,
final Tokenizer.Cursor
defines the bounds and current position of the buffer
cursor
)

Skips semantically insignificant whitespace characters and moves the cursor to the closest non-whitespace character.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

CRback to summary
public static final int CR
DQUOTEback to summary
public static final char DQUOTE

Double quote

ESCAPEback to summary
public static final char ESCAPE

Backward slash / escape character

HTback to summary
public static final int HT
INSTANCEback to summary
public static final Tokenizer INSTANCE
LFback to summary
public static final int LF
SPback to summary
public static final int SP

Constructor Detail

Tokenizerback to summary
public Tokenizer()

Method Detail

copyContentback to summary
public void copyContent(final CharSequence buf, final Tokenizer.Cursor cursor, final BitSet delimiters, final StringBuilder dst)

Transfers content into the destination buffer until a whitespace character or any of the given delimiters is encountered.

Parameters
buf:CharSequence

buffer with the sequence of chars to be parsed

cursor:Tokenizer.Cursor

defines the bounds and current position of the buffer

delimiters:BitSet

set of delimiting characters. Can be null if the value is delimited by a whitespace only.

dst:StringBuilder

destination buffer

copyQuotedContentback to summary
public void copyQuotedContent(final CharSequence buf, final Tokenizer.Cursor cursor, final StringBuilder dst)

Transfers content enclosed with quote marks into the destination buffer.

Parameters
buf:CharSequence

buffer with the sequence of chars to be parsed

cursor:Tokenizer.Cursor

defines the bounds and current position of the buffer

dst:StringBuilder

destination buffer

copyUnquotedContentback to summary
public void copyUnquotedContent(final CharSequence buf, final Tokenizer.Cursor cursor, final BitSet delimiters, final StringBuilder dst)

Transfers content into the destination buffer until a whitespace character, a quote, or any of the given delimiters is encountered.

Parameters
buf:CharSequence

buffer with the sequence of chars to be parsed

cursor:Tokenizer.Cursor

defines the bounds and current position of the buffer

delimiters:BitSet

set of delimiting characters. Can be null if the value is delimited by a whitespace or a quote only.

dst:StringBuilder

destination buffer

INIT_BITSETback to summary
public static BitSet INIT_BITSET(final int... b)
isWhitespaceback to summary
public static boolean isWhitespace(final char ch)
parseContentback to summary
public String parseContent(final CharSequence buf, final Tokenizer.Cursor cursor, final BitSet delimiters)

Extracts from the sequence of chars a token terminated with any of the given delimiters or a whitespace characters.

Parameters
buf:CharSequence

buffer with the sequence of chars to be parsed

cursor:Tokenizer.Cursor

defines the bounds and current position of the buffer

delimiters:BitSet

set of delimiting characters. Can be null if the token is not delimited by any character.

parseTokenback to summary
public String parseToken(final CharSequence buf, final Tokenizer.Cursor cursor, final BitSet delimiters)

Extracts from the sequence of chars a token terminated with any of the given delimiters discarding semantically insignificant whitespace characters.

Parameters
buf:CharSequence

buffer with the sequence of chars to be parsed

cursor:Tokenizer.Cursor

defines the bounds and current position of the buffer

delimiters:BitSet

set of delimiting characters. Can be null if the token is not delimited by any character.

parseValueback to summary
public String parseValue(final CharSequence buf, final Tokenizer.Cursor cursor, final BitSet delimiters)

Extracts from the sequence of chars a value which can be enclosed in quote marks and terminated with any of the given delimiters discarding semantically insignificant whitespace characters.

Parameters
buf:CharSequence

buffer with the sequence of chars to be parsed

cursor:Tokenizer.Cursor

defines the bounds and current position of the buffer

delimiters:BitSet

set of delimiting characters. Can be null if the value is not delimited by any character.

skipWhiteSpaceback to summary
public void skipWhiteSpace(final CharSequence buf, final Tokenizer.Cursor cursor)

Skips semantically insignificant whitespace characters and moves the cursor to the closest non-whitespace character.

Parameters
buf:CharSequence

buffer with the sequence of chars to be parsed

cursor:Tokenizer.Cursor

defines the bounds and current position of the buffer

org.apache.hc.core5.util back to summary

public Class Tokenizer.Cursor

extends Object
Class Inheritance
Known Direct Subclasses
org.apache.hc.core5.http.message.ParserCursor

Field Summary

Modifier and TypeField and Description
private final int
private int
private final int

Constructor Summary

AccessConstructor and Description
public
Cursor(final int lowerBound, final int upperBound)

Method Summary

Modifier and TypeMethod and Description
public boolean
public int
public int
public int
public String
toString()

Overrides java.lang.Object.toString.

Returns a string representation of the object.
public void
updatePos(final int pos)

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAllwaitwaitwait

Field Detail

lowerBoundback to summary
private final int lowerBound
posback to summary
private int pos
upperBoundback to summary
private final int upperBound

Constructor Detail

Cursorback to summary
public Cursor(final int lowerBound, final int upperBound)

Method Detail

atEndback to summary
public boolean atEnd()
getLowerBoundback to summary
public int getLowerBound()
getPosback to summary
public int getPos()
getUpperBoundback to summary
public int getUpperBound()
toStringback to summary
public String toString()

Overrides java.lang.Object.toString.

Doc from java.lang.Object.toString.

Returns a string representation of the object.

Returns:String

a string representation of the object.

Annotations
@Override
updatePosback to summary
public void updatePos(final int pos)