Top Description Fields Constructors Methods
com.rometools.rome.io

public Class XmlReader

extends Reader
Class Inheritance
Imports
java.io.BufferedInputStream, .File, .FileInputStream, .IOException, .InputStream, .InputStreamReader, .Reader, java.net.HttpURLConnection, .URL, .URLConnection, java.text.MessageFormat, java.util.Locale, .Map, .Map.Entry, java.util.regex.Matcher, .Pattern

Character stream that handles (or at least attemtps to) all the necessary Voodo to figure out the charset encoding of the XML document within the stream.

Important Note

This class is not related in any way to the org.xml.sax.XMLReader. This one IS a character stream.

All this has to be done without consuming characters from the stream, if not the XML parser will not recognized the document as a valid XML. This is not 100% true, but it's close enough (UTF-8 BOM is not handled by all parsers right now, XmlReader handles it and things work in all parsers).

The XmlReader class handles the charset encoding of XML documents in Files, raw streams and HTTP streams by offering a wide set of constructors.

By default the charset encoding detection is lenient, the constructor with the lenient flag can be used for an script (following HTTP MIME and XML specifications). All this is nicely explained by Mark Pilgrim in his blog, Determining the character encoding of a feed.

Field Summary

Modifier and TypeField and Description
private static final int
private static final Pattern
private static final String
private final String
private String
private static final Pattern
private static final MessageFormat
private static final MessageFormat
private static final MessageFormat
private static final MessageFormat
private static final MessageFormat
private Reader
private static String
private static final String
private static final String
private static final String
private static final String
private static final String
Inherited from java.io.Reader:
lock

Constructor Summary

AccessConstructor and Description
public
XmlReader(final File
File to create a Reader from.
file
)

Creates a Reader for a File.

public
XmlReader(final InputStream
InputStream to create a Reader from.
is
)

Creates a Reader for a raw InputStream.

public
XmlReader(final InputStream
InputStream to create a Reader from.
is
,
final boolean
indicates if the charset encoding detection should be relaxed.
lenient
,
final String
default encoding to use if one cannot be detected.
defaultEncoding
)

Creates a Reader for a raw InputStream and uses the provided default encoding if none is determined.

public
XmlReader(final InputStream
InputStream to create a Reader from.
is
,
final boolean
indicates if the charset encoding detection should be relaxed.
lenient
)

Creates a Reader for a raw InputStream.

public
XmlReader(final URL
URL to create a Reader from.
url
)

Deprecated As of release 2.0.0 and will be removed in version 3.0.0 due to too many problems with this approach.
Creates a Reader using the InputStream of a URL.
public
XmlReader(final URL
URL to create a Reader from.
url
,
final Map<String, String>
optional Map of headers to set on http request.
requestHeaders
)

Deprecated As of release 2.0.0 and will be removed in version 3.0.0 due to too many problems with this approach.
Creates a Reader using the InputStream of a URL.
public
XmlReader(final URLConnection
URLConnection to create a Reader from.
conn
)

Deprecated As of release 2.0.0 and will be removed in version 3.0.0 due to too many problems with this approach.
Creates a Reader using the InputStream of a URLConnection.
public
XmlReader(final URLConnection
URLConnection to create a Reader from.
conn
,
final Map<String, String>
optional Map of headers to set on http request.
requestHeaders
)

Deprecated As of release 2.0.0 and will be removed in version 3.0.0 due to too many problems with this approach.
Creates a Reader using the InputStream of a URLConnection.
public
XmlReader(final InputStream
InputStream to create the reader from.
is
,
final String
content-type header to use for the resolution of the charset encoding.
httpContentType
)

Creates a Reader using an InputStream and the associated content-type header.

public
XmlReader(final InputStream
InputStream to create the reader from.
is
,
final String
content-type header to use for the resolution of the charset encoding.
httpContentType
,
final boolean
indicates if the charset encoding detection should be relaxed.
lenient
,
final String
default encoding to use if one cannot be detected.
defaultEncoding
)

Creates a Reader using an InputStream and the associated content-type header.

public
XmlReader(final InputStream
InputStream to create the reader from.
is
,
final String
content-type header to use for the resolution of the charset encoding.
httpContentType
,
final boolean
indicates if the charset encoding detection should be relaxed.
lenient
)

Creates a Reader using an InputStream and the associated content-type header.

Method Summary

Modifier and TypeMethod and Description
private String
calculateHttpEncoding(final String cTMime, final String cTEnc, final String bomEnc, final String xmlGuessEnc, final String xmlEnc, final InputStream is, final boolean lenient)

private String
calculateRawEncoding(final String bomEnc, final String xmlGuessEnc, final String xmlEnc, final InputStream is)

public void
close()

Implements abstract java.io.Reader.close.

Implements java.io.Closeable.close.

Closes the XmlReader stream.
private void
doHttpStream(final InputStream is, final String httpContentType, final boolean lenient)

private void
private void
doRawStream(final InputStream is, final boolean lenient)

private static String
private static String
getContentTypeEncoding(final String httpContentType)

private static String
getContentTypeMime(final String httpContentType)

public static String

Returns:

the default encoding to use.
getDefaultEncoding
()

Returns the default encoding to use if none is set in HTTP content-type, XML prolog and the rules based on content-type are not adequate.

public String

Returns:

charset encoding.
getEncoding
()

Returns the charset encoding of the XmlReader.

private static String
pack-priv static String
getXmlProlog(final InputStream is, final String guessedEnc)

private static boolean
isAppXml(final String mime)

private static boolean
isTextXml(final String mime)

private void
prepareReader(final InputStream is, final String encoding)

public int
read(final char[]
Destination buffer
buf
,
final int
Offset at which to start storing characters
offset
,
final int
Maximum number of characters to read
len
)

Implements abstract java.io.Reader.read.

Reads characters into a portion of an array.
public static void
setDefaultEncoding(final String
charset encoding to default to.
encoding
)

Sets the default encoding to use if none is set in HTTP content-type, XML prolog and the rules based on content-type are not adequate.

private void
setRequestHeader(final URLConnection conn, final Map<String, String> requestHeaders)

Inherited from java.io.Reader:
markmarkSupportednullReaderreadreadreadreadyresetskiptransferTo

Field Detail

BUFFER_SIZEback to summary
private static final int BUFFER_SIZE
CHARSET_PATTERNback to summary
private static final Pattern CHARSET_PATTERN
CP1047back to summary
private static final String CP1047
defaultEncodingback to summary
private final String defaultEncoding
encodingback to summary
private String encoding
ENCODING_PATTERNback to summary
private static final Pattern ENCODING_PATTERN
HTTP_EX_1back to summary
private static final MessageFormat HTTP_EX_1
HTTP_EX_2back to summary
private static final MessageFormat HTTP_EX_2
HTTP_EX_3back to summary
private static final MessageFormat HTTP_EX_3
RAW_EX_1back to summary
private static final MessageFormat RAW_EX_1
RAW_EX_2back to summary
private static final MessageFormat RAW_EX_2
readerback to summary
private Reader reader
staticDefaultEncodingback to summary
private static String staticDefaultEncoding
US_ASCIIback to summary
private static final String US_ASCII
UTF_16back to summary
private static final String UTF_16
UTF_16BEback to summary
private static final String UTF_16BE
UTF_16LEback to summary
private static final String UTF_16LE
UTF_8back to summary
private static final String UTF_8

Constructor Detail

XmlReaderback to summary
public XmlReader(final File file) throws IOException

Creates a Reader for a File.

It looks for the UTF-8 BOM first, if none sniffs the XML prolog charset, if this is also missing defaults to UTF-8.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters
file:File

File to create a Reader from.

Exceptions
IOException:
thrown if there is a problem reading the file.
XmlReaderback to summary
public XmlReader(final InputStream is) throws IOException

Creates a Reader for a raw InputStream.

It follows the same logic used for files.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters
is:InputStream

InputStream to create a Reader from.

Exceptions
IOException:
thrown if there is a problem reading the stream.
XmlReaderback to summary
public XmlReader(final InputStream is, final boolean lenient, final String defaultEncoding) throws IOException, XmlReaderException

Creates a Reader for a raw InputStream and uses the provided default encoding if none is determined.

It follows the same logic used for files.

If lenient detection is indicated and the detection above fails as per specifications it then attempts the following:

If the content type was 'text/html' it replaces it with 'text/xml' and tries the detection again.

Else if the XML prolog had a charset encoding that encoding is used.

Else if the content type had a charset encoding that encoding is used.

Else 'UTF-8' is used.

If lenient detection is indicated an XmlReaderException is never thrown.

Parameters
is:InputStream

InputStream to create a Reader from.

lenient:boolean

indicates if the charset encoding detection should be relaxed.

defaultEncoding:String

default encoding to use if one cannot be detected.

Exceptions
IOException:
thrown if there is a problem reading the stream.
XmlReaderException:
thrown if the charset encoding could not be determined according to the specs.
XmlReaderback to summary
public XmlReader(final InputStream is, final boolean lenient) throws IOException, XmlReaderException

Creates a Reader for a raw InputStream.

It follows the same logic used for files.

If lenient detection is indicated and the detection above fails as per specifications it then attempts the following:

If the content type was 'text/html' it replaces it with 'text/xml' and tries the detection again.

Else if the XML prolog had a charset encoding that encoding is used.

Else if the content type had a charset encoding that encoding is used.

Else 'UTF-8' is used.

If lenient detection is indicated an XmlReaderException is never thrown.

Parameters
is:InputStream

InputStream to create a Reader from.

lenient:boolean

indicates if the charset encoding detection should be relaxed.

Exceptions
IOException:
thrown if there is a problem reading the stream.
XmlReaderException:
thrown if the charset encoding could not be determined according to the specs.
XmlReaderback to summary
public XmlReader(final URL url) throws IOException

Deprecated

As of release 2.0.0 and will be removed in version 3.0.0 due to too many problems with this approach.

Creates a Reader using the InputStream of a URL.

If the URL is not of type HTTP and there is not 'content-type' header in the fetched data it uses the same logic used for Files.

If the URL is a HTTP Url or there is a 'content-type' header in the fetched data it uses the same logic used for an InputStream with content-type.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters
url:URL

URL to create a Reader from.

Annotations
@Deprecated
Exceptions
IOException:
thrown if there is a problem reading the stream of the URL.
XmlReaderback to summary
public XmlReader(final URL url, final Map<String, String> requestHeaders) throws IOException

Deprecated

As of release 2.0.0 and will be removed in version 3.0.0 due to too many problems with this approach.

Creates a Reader using the InputStream of a URL.

If the URL is not of type HTTP and there is not 'content-type' header in the fetched data it uses the same logic used for Files.

If the URL is a HTTP Url or there is a 'content-type' header in the fetched data it uses the same logic used for an InputStream with content-type.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters
url:URL

URL to create a Reader from.

requestHeaders:Map<String, String>

optional Map of headers to set on http request.

Annotations
@Deprecated
Exceptions
IOException:
thrown if there is a problem reading the stream of the URL.
XmlReaderback to summary
public XmlReader(final URLConnection conn) throws IOException

Deprecated

As of release 2.0.0 and will be removed in version 3.0.0 due to too many problems with this approach.

Creates a Reader using the InputStream of a URLConnection.

If the URLConnection is not of type HttpURLConnection and there is not 'content-type' header in the fetched data it uses the same logic used for files.

If the URLConnection is a HTTP Url or there is a 'content-type' header in the fetched data it uses the same logic used for an InputStream with content-type.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters
conn:URLConnection

URLConnection to create a Reader from.

Annotations
@Deprecated
Exceptions
IOException:
thrown if there is a problem reading the stream of the URLConnection.
XmlReaderback to summary
public XmlReader(final URLConnection conn, final Map<String, String> requestHeaders) throws IOException

Deprecated

As of release 2.0.0 and will be removed in version 3.0.0 due to too many problems with this approach.

Creates a Reader using the InputStream of a URLConnection.

If the URLConnection is not of type HttpURLConnection and there is not 'content-type' header in the fetched data it uses the same logic used for files.

If the URLConnection is a HTTP Url or there is a 'content-type' header in the fetched data it uses the same logic used for an InputStream with content-type.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters
conn:URLConnection

URLConnection to create a Reader from.

requestHeaders:Map<String, String>

optional Map of headers to set on http request.

Annotations
@Deprecated
Exceptions
IOException:
thrown if there is a problem reading the stream of the URLConnection.
XmlReaderback to summary
public XmlReader(final InputStream is, final String httpContentType) throws IOException

Creates a Reader using an InputStream and the associated content-type header.

First it checks if the stream has BOM. If there is not BOM checks the content-type encoding. If there is not content-type encoding checks the XML prolog encoding. If there is not XML prolog encoding uses the default encoding mandated by the content-type MIME type.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters
is:InputStream

InputStream to create the reader from.

httpContentType:String

content-type header to use for the resolution of the charset encoding.

Exceptions
IOException:
thrown if there is a problem reading the file.
XmlReaderback to summary
public XmlReader(final InputStream is, final String httpContentType, final boolean lenient, final String defaultEncoding) throws IOException, XmlReaderException

Creates a Reader using an InputStream and the associated content-type header.

First it checks if the stream has BOM. If there is not BOM checks the content-type encoding. If there is not content-type encoding checks the XML prolog encoding. If there is not XML prolog encoding uses the default encoding mandated by the content-type MIME type.

If lenient detection is indicated and the detection above fails as per specifications it then attempts the following:

If the content type was 'text/html' it replaces it with 'text/xml' and tries the detection again.

Else if the XML prolog had a charset encoding that encoding is used.

Else if the content type had a charset encoding that encoding is used.

Else 'UTF-8' is used.

If lenient detection is indicated and XmlReaderException is never thrown.

Parameters
is:InputStream

InputStream to create the reader from.

httpContentType:String

content-type header to use for the resolution of the charset encoding.

lenient:boolean

indicates if the charset encoding detection should be relaxed.

defaultEncoding:String

default encoding to use if one cannot be detected.

Exceptions
IOException:
thrown if there is a problem reading the file.
XmlReaderException:
thrown if the charset encoding could not be determined according to the specs.
XmlReaderback to summary
public XmlReader(final InputStream is, final String httpContentType, final boolean lenient) throws IOException, XmlReaderException

Creates a Reader using an InputStream and the associated content-type header.

First it checks if the stream has BOM. If there is not BOM checks the content-type encoding. If there is not content-type encoding checks the XML prolog encoding. If there is not XML prolog encoding uses the default encoding mandated by the content-type MIME type.

If lenient detection is indicated and the detection above fails as per specifications it then attempts the following:

If the content type was 'text/html' it replaces it with 'text/xml' and tries the detection again.

Else if the XML prolog had a charset encoding that encoding is used.

Else if the content type had a charset encoding that encoding is used.

Else 'UTF-8' is used.

If lenient detection is indicated and XmlReaderException is never thrown.

Parameters
is:InputStream

InputStream to create the reader from.

httpContentType:String

content-type header to use for the resolution of the charset encoding.

lenient:boolean

indicates if the charset encoding detection should be relaxed.

Exceptions
IOException:
thrown if there is a problem reading the file.
XmlReaderException:
thrown if the charset encoding could not be determined according to the specs.

Method Detail

calculateHttpEncodingback to summary
private String calculateHttpEncoding(final String cTMime, final String cTEnc, final String bomEnc, final String xmlGuessEnc, final String xmlEnc, final InputStream is, final boolean lenient) throws IOException
calculateRawEncodingback to summary
private String calculateRawEncoding(final String bomEnc, final String xmlGuessEnc, final String xmlEnc, final InputStream is) throws IOException
closeback to summary
public void close() throws IOException

Implements abstract java.io.Reader.close.

Implements java.io.Closeable.close.

Closes the XmlReader stream.

Annotations
@Override
Exceptions
IOException:
thrown if there was a problem closing the stream.
doHttpStreamback to summary
private void doHttpStream(final InputStream is, final String httpContentType, final boolean lenient) throws IOException
doLenientDetectionback to summary
private void doLenientDetection(String httpContentType, XmlReaderException ex) throws IOException
doRawStreamback to summary
private void doRawStream(final InputStream is, final boolean lenient) throws IOException
getBOMEncodingback to summary
private static String getBOMEncoding(final BufferedInputStream is) throws IOException
getContentTypeEncodingback to summary
private static String getContentTypeEncoding(final String httpContentType)
getContentTypeMimeback to summary
private static String getContentTypeMime(final String httpContentType)
getDefaultEncodingback to summary
public static String getDefaultEncoding()

Returns the default encoding to use if none is set in HTTP content-type, XML prolog and the rules based on content-type are not adequate.

If it is NULL the content-type based rules are used.

Returns:String

the default encoding to use.

getEncodingback to summary
public String getEncoding()

Returns the charset encoding of the XmlReader.

Returns:String

charset encoding.

getXMLGuessEncodingback to summary
private static String getXMLGuessEncoding(final BufferedInputStream is) throws IOException
getXmlPrologback to summary
pack-priv static String getXmlProlog(final InputStream is, final String guessedEnc) throws IOException
isAppXmlback to summary
private static boolean isAppXml(final String mime)
isTextXmlback to summary
private static boolean isTextXml(final String mime)
prepareReaderback to summary
private void prepareReader(final InputStream is, final String encoding) throws IOException
readback to summary
public int read(final char[] buf, final int offset, final int len) throws IOException

Implements abstract java.io.Reader.read.

Doc from java.io.Reader.read.

Reads characters into a portion of an array. This method will block until some input is available, an I/O error occurs, or the end of the stream is reached.

If len is zero, then no characters are read and 0 is returned; otherwise, there is an attempt to read at least one character. If no character is available because the stream is at its end, the value -1 is returned; otherwise, at least one character is read and stored into cbuf.

Parameters
buf:char[]

Destination buffer

offset:int

Offset at which to start storing characters

len:int

Maximum number of characters to read

Returns:int

The number of characters read, or -1 if the end of the stream has been reached

Annotations
@Override
Exceptions
IOException:
If an I/O error occurs
setDefaultEncodingback to summary
public static void setDefaultEncoding(final String encoding)

Sets the default encoding to use if none is set in HTTP content-type, XML prolog and the rules based on content-type are not adequate.

If it is set to NULL the content-type based rules are used.

By default it is NULL.

Parameters
encoding:String

charset encoding to default to.

setRequestHeaderback to summary
private void setRequestHeader(final URLConnection conn, final Map<String, String> requestHeaders)