CharsetDetector provides a facility for detecting the
charset or encoding of character data in an unknown format.
The input data can either be from an input stream or an array of bytes.
The result of the detection operation is a list of possibly matching
charsets, or, for simple use, you can just ask for a Java Reader that
will will work over the input data.
Character set detection is at best an imprecise operation. The detection process will attempt to identify the charset that best matches the characteristics of the byte data, but the process is partly statistical in nature, and the results can not be guaranteed to always be correct.
For best accuracy in charset detection, the input data should be primarily in a single language, and a minimum of a few hundred bytes worth of plain text in the language are needed. The detection process will attempt to ignore html or xml style markup that could otherwise obscure the content.
| Modifier and Type | Class and Description |
|---|---|
| private static class |
| Modifier and Type | Field and Description |
|---|---|
| private static final List | |
| pack-priv short[] | |
| pack-priv boolean | |
| pack-priv String | |
| private boolean[] | |
| pack-priv byte[] | |
| pack-priv int | |
| pack-priv InputStream | |
| pack-priv byte[] | |
| pack-priv int | |
| private boolean | |
| private static final int |
| Access | Constructor and Description |
|---|---|
| public |
| Modifier and Type | Method and Description |
|---|---|
| public CharsetMatch | Returns: a CharsetMatch object representing the best matching charset, ornull if there are no matches.Return the charset that best matches the supplied input data. |
| public CharsetMatch[] | Returns: An array of CharsetMatch objects representing possibly matching charsets.Return an array of all charsets that appear to be plausible matches with the input data. |
| public boolean | Returns: The previous setting.true to enable input text filtering.Enable filtering of input text. |
| public static String[] | Returns: an array of the names of all charsets supported byCharsetDetector class.Get the names of all charsets supported by |
| public String[] | Returns: an array of the names of charsets that can be recognized by this CharsetDetector instance.
Deprecated
This API is ICU internal only.
Get the names of charsets that can be recognized by this CharsetDetector instance.
|
| public Reader | getReader(InputStream
The source of the byte data in the unknown charset. in, String A declared encoding for the data, if available,
or null or an empty string if none is available. declaredEncoding)Autodetect the charset of an inputStream, and return a Java Reader to access the converted input data. |
| public String | |
| public boolean | Returns: true if input text will be filtered.Test whether or not input filtering is enabled. |
| private void | |
| public CharsetDetector | setDeclaredEncoding(String
The declared encoding encoding)Set the declared encoding for charset detection. |
| public CharsetDetector | Returns: A reference to thisCharsetDetector.the name of charset encoding. encoding, boolean true to enable, or false to disable the
charset encoding.
Deprecated
This API is ICU internal only.
Enable or disable individual charset encoding.
|
| public CharsetDetector | Returns: This CharsetDetectorthe input text of unknown encoding in)Set the input text (byte) data whose charset is to be detected. |
| public CharsetDetector | Returns: This CharsetDetectorthe input text of unknown encoding in)Set the input text (byte) data whose charset is to be detected. |
| ALL_CS_RECOGNIZERS | back to summary |
|---|---|
| private static final List<CharsetDetector. | |
| fByteStats | back to summary |
|---|---|
| pack-priv short[] fByteStats | |
| fC1Bytes | back to summary |
|---|---|
| pack-priv boolean fC1Bytes | |
| fDeclaredEncoding | back to summary |
|---|---|
| pack-priv String fDeclaredEncoding | |
| fEnabledRecognizers | back to summary |
|---|---|
| private boolean[] fEnabledRecognizers | |
| fInputBytes | back to summary |
|---|---|
| pack-priv byte[] fInputBytes | |
| fInputLen | back to summary |
|---|---|
| pack-priv int fInputLen | |
| fInputStream | back to summary |
|---|---|
| pack-priv InputStream fInputStream | |
| fRawInput | back to summary |
|---|---|
| pack-priv byte[] fRawInput | |
| fRawLength | back to summary |
|---|---|
| pack-priv int fRawLength | |
| fStripTags | back to summary |
|---|---|
| private boolean fStripTags | |
| kBufSize | back to summary |
|---|---|
| private static final int kBufSize | |
| CharsetDetector | back to summary |
|---|---|
| public CharsetDetector() Constructor
| |
| detect | back to summary |
|---|---|
| public CharsetMatch detect() Return the charset that best matches the supplied input data. Note though, that because the detection only looks at the start of the input data, there is a possibility that the returned charset will fail to handle the full set of input data. Raise an exception if
| |
| detectAll | back to summary |
|---|---|
| public CharsetMatch[] detectAll() Return an array of all charsets that appear to be plausible matches with the input data. The array is ordered with the best quality match first. Raise an exception if
| |
| enableInputFilter | back to summary |
|---|---|
| public boolean enableInputFilter(boolean filter) Enable filtering of input text. If filtering is enabled, text within angle brackets ("<" and ">") will be removed before detection.
| |
| getAllDetectableCharsets | back to summary |
|---|---|
| public static String[] getAllDetectableCharsets() Get the names of all charsets supported by Note Multiple different charset encodings in a same family may use a single shared name in this implementation. For example, this method returns an array including "ISO-8859-1" (ISO Latin 1), but not including "windows-1252" (Windows Latin 1). However, actual detection result could be "windows-1252" when the input data matches Latin 1 code points with any points only available in "windows-1252". | |
| getDetectableCharsets | back to summary |
|---|---|
| public String[] getDetectableCharsets()
Deprecated This API is ICU internal only. Get the names of charsets that can be recognized by this CharsetDetector instance. | |
| getReader | back to summary |
|---|---|
| public Reader getReader(InputStream in, String declaredEncoding) Autodetect the charset of an inputStream, and return a Java Reader to access the converted input data.
This is a convenience method that is equivalent to
For the input stream that supplies the character data, markSupported() must be true; the charset detection will read a small amount of data, then return the stream to its original position via the InputStream.reset() operation. The exact amount that will be read depends on the characteristics of the data itself. Raise an exception if no charsets appear to match the input data.
| |
| getString | back to summary |
|---|---|
| public String getString(byte[] in, String declaredEncoding) Autodetect the charset of an inputStream, and return a String containing the converted input data.
This is a convenience method that is equivalent to
Raise an exception if no charsets appear to match the input data. | |
| inputFilterEnabled | back to summary |
|---|---|
| public boolean inputFilterEnabled() Test whether or not input filtering is enabled.
| |
| MungeInput | back to summary |
|---|---|
| private void MungeInput() | |
| setDeclaredEncoding | back to summary |
|---|---|
| public CharsetDetector setDeclaredEncoding(String encoding) Set the declared encoding for charset detection. The declared encoding of an input text is an encoding obtained from an http header or xml declaration or similar source that can be provided as additional information to the charset detector. A match between a declared encoding and a possible detected encoding will raise the quality of that detected encoding by a small delta, and will also appear as a "reason" for the match. A declared encoding that is incompatible with the input data being analyzed will not be added to the list of possible encodings. | |
| setDetectableCharset | back to summary |
|---|---|
| public CharsetDetector setDetectableCharset(String encoding, boolean enabled)
Deprecated This API is ICU internal only. Enable or disable individual charset encoding.
A name of charset encoding must be included in the names returned by
| |
| setText | back to summary |
|---|---|
| public CharsetDetector setText(byte[] in) Set the input text (byte) data whose charset is to be detected.
| |
| setText | back to summary |
|---|---|
| public CharsetDetector setText(InputStream in) throws IOException Set the input text (byte) data whose charset is to be detected. The input stream that supplies the character data must have markSupported() == true; the charset detection process will read a small amount of data, then return the stream to its original position via the InputStream.reset() operation. The exact amount that will be read depends on the characteristics of the data itself.
| |
| Modifier and Type | Field and Description |
|---|---|
| pack-priv boolean | |
| pack-priv CharsetRecognizer |
| Access | Constructor and Description |
|---|---|
| pack-priv |
| isDefaultEnabled | back to summary |
|---|---|
| pack-priv boolean isDefaultEnabled | |
| recognizer | back to summary |
|---|---|
| pack-priv CharsetRecognizer recognizer | |
| CSRecognizerInfo | back to summary |
|---|---|
| pack-priv CSRecognizerInfo(CharsetRecognizer recognizer, boolean isDefaultEnabled) | |