Top Description Inners Fields Constructors Methods
org.eclipse.jgit.transport

public Class AmazonS3

extends Object
Class Inheritance
Static Imports
java.nio.charset.StandardCharsets.UTF_8

A simple HTTP REST client for the Amazon S3 service.

This client uses the REST API to communicate with the Amazon S3 servers and read or write content through a bucket that the user has access to. It is a very lightweight implementation of the S3 API and therefore does not have all of the bells and whistles of popular client implementations.

Authentication is always performed using the user's AWSAccessKeyId and their private AWSSecretAccessKey.

Optional client-side encryption may be enabled if requested. The format is compatible with jets3t, a popular Java based Amazon S3 client library. Enabling encryption can hide sensitive data from the operators of the S3 service.

Nested and Inner Type Summary

Modifier and TypeClass and Description
private static class
AmazonS3.KeyInfo

KeyInfo enables sorting of keys by lastModified time

pack-priv static interface
AmazonS3.Keys

Property names used in amazon connection configuration file.

private class

Field Summary

Modifier and TypeField and Description
private final String
acl

ACL to apply to created objects.

private static final String
private static final String
private static final String
private final String
awsApiSignatureVersion

AWS API Signature Version.

private final String
domain

S3 Bucket Domain.

private final WalkEncryption
encryption

Encryption algorithm, may be a null instance that provides pass-through.

private static final String
pack-priv final int
maxAttempts

Maximum number of times to try an operation.

private final String
protocol

S3 Protocol, "https" or "http"; defaults to "http".

private final ProxySelector
proxySelector

Our HTTP proxy support, in case we are behind a firewall.

private final String
publicKey

AWSAccessKeyId, public string that identifies the user's account.

private final String
region

S3 Region.

private final char[]
secretKey

AWSSecretAccessKey, private string used to access a user's account.

private final SecretKeySpec
secretKeySpec

Decoded form of the private AWSSecretAccessKey, to sign requests.

private static final Set<String>
private final File
tmpDir

Directory for locally buffered content.

private static final String
private static final String

Constructor Summary

AccessConstructor and Description
public
AmazonS3(final Properties
connection properties.
props
)

Create a new S3 client for the supplied user information.

Method Summary

Modifier and TypeMethod and Description
pack-priv void
authorize(HttpURLConnection httpURLConnection, Map<String, String> queryParams, long contentLength, final String bodyHash)

pack-priv void
public OutputStream

Returns:

a stream which accepts the new data, and transmits once closed.
beginPut
(final String
name of the bucket storing the object.
bucket
,
final String
key of the object within its bucket.
key
,
final ProgressMonitor
(optional) progress monitor to post upload completion to during the stream's close method.
monitor
,
final String
(optional) task name to display during the close method.
monitorTask
)

Atomically create or replace a single large object.

public InputStream

Returns:

stream to read plain text from.
decrypt
(URLConnection
connection previously created by get(String, String)}.
u
)

Decrypt an input stream from get(String, String).

public void
delete(String
name of the bucket storing the object.
bucket
,
String
key of the object within its bucket.
key
)

Delete a single object.

pack-priv IOException
error(final String action, final String key, final HttpURLConnection c)

public URLConnection

Returns:

connection to stream the content of the object. The request properties of the connection may not be modified by the caller as the request parameters have already been signed.
get
(String
name of the bucket storing the object.
bucket
,
String
key of the object within its bucket.
key
)

Get the content of a bucket object.

private static String
private static boolean
public List<String>

Returns:

list of keys starting with prefix, after removing prefix (or prefix + "/")from all of them.
list
(String
name of the bucket whose objects should be listed.
bucket
,
String
common prefix to filter the results by. Must not be null. Supplying the empty string will list all keys in the bucket. Supplying a non-empty string will act as though a trailing '/' appears in prefix, even if it does not.
prefix
)

List the names of keys available within a bucket.

pack-priv IOException
maxAttempts(String action, String key)

private static MessageDigest
private HttpURLConnection
open(final String method, final String bucket, final String key)

pack-priv HttpURLConnection
open(final String method, final String bucket, final String key, final Map<String, String> args)

pack-priv static Properties
properties(File authFile)

public void
put(String
name of the bucket storing the object.
bucket
,
String
key of the object within its bucket.
key
,
byte[]
new data content for the object. Must not be null. Zero length array will create a zero length object.
data
)

Atomically create or replace a single small object.

pack-priv void
putImpl(final String bucket, final String key, final byte[] csum, final TemporaryBuffer buf, ProgressMonitor monitor, String monitorTask)

private static String
private static String
Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

aclback to summary
private final String acl

ACL to apply to created objects.

AWS_API_V2back to summary
private static final String AWS_API_V2
AWS_API_V4back to summary
private static final String AWS_API_V4
AWS_S3_SERVICE_NAMEback to summary
private static final String AWS_S3_SERVICE_NAME
awsApiSignatureVersionback to summary
private final String awsApiSignatureVersion

AWS API Signature Version.

domainback to summary
private final String domain

S3 Bucket Domain.

encryptionback to summary
private final WalkEncryption encryption

Encryption algorithm, may be a null instance that provides pass-through.

HMACback to summary
private static final String HMAC
maxAttemptsback to summary
pack-priv final int maxAttempts

Maximum number of times to try an operation.

protocolback to summary
private final String protocol

S3 Protocol, "https" or "http"; defaults to "http".

proxySelectorback to summary
private final ProxySelector proxySelector

Our HTTP proxy support, in case we are behind a firewall.

publicKeyback to summary
private final String publicKey

AWSAccessKeyId, public string that identifies the user's account.

regionback to summary
private final String region

S3 Region.

secretKeyback to summary
private final char[] secretKey

AWSSecretAccessKey, private string used to access a user's account.

secretKeySpecback to summary
private final SecretKeySpec secretKeySpec

Decoded form of the private AWSSecretAccessKey, to sign requests.

SIGNED_HEADERSback to summary
private static final Set<String> SIGNED_HEADERS
tmpDirback to summary
private final File tmpDir

Directory for locally buffered content.

X_AMZ_ACLback to summary
private static final String X_AMZ_ACL
X_AMZ_METAback to summary
private static final String X_AMZ_META

Constructor Detail

AmazonS3back to summary
public AmazonS3(final Properties props)

Create a new S3 client for the supplied user information.

The connection properties are a subset of those supported by the popular jets3t library. For example:

# AWS API signature version, must be one of:
#   2 - deprecated (not supported in all AWS regions)
#   4 - latest (supported in all AWS regions)
# Defaults to 2.
aws.api.signature.version: 4

# AWS Access and Secret Keys (required)
accesskey: <YourAWSAccessKey>
secretkey: <YourAWSSecretKey>

# Access Control List setting to apply to uploads, must be one of:
# PRIVATE, PUBLIC_READ (defaults to PRIVATE).
acl: PRIVATE

# S3 Domain
# AWS S3 Region Domain (defaults to s3.amazonaws.com)
domain: s3.amazonaws.com

# AWS S3 Region (required if aws.api.signature.version = 4)
region: us-west-2

# Number of times to retry after internal error from S3.
httpclient.retry-max: 3

# End-to-end encryption (hides content from S3 owners)
password: <encryption pass-phrase>
crypto.algorithm: PBEWithMD5AndDES
Parameters
props:Properties

connection properties.

Method Detail

authorizeback to summary
pack-priv void authorize(HttpURLConnection httpURLConnection, Map<String, String> queryParams, long contentLength, final String bodyHash) throws IOException
authorizeV2back to summary
pack-priv void authorizeV2(HttpURLConnection c) throws IOException
beginPutback to summary
public OutputStream beginPut(final String bucket, final String key, final ProgressMonitor monitor, final String monitorTask) throws IOException

Atomically create or replace a single large object.

Initially the returned output stream buffers data into memory, but if the total number of written bytes starts to exceed an internal limit the data is spooled to a temporary file on the local drive.

Network transmission is attempted only when close() gets called at the end of output. Closing the returned stream can therefore take significant time, especially if the written content is very large.

End-to-end data integrity is assured by internally computing the MD5 checksum of the supplied data and transmitting the checksum along with the data itself.

Parameters
bucket:String

name of the bucket storing the object.

key:String

key of the object within its bucket.

monitor:ProgressMonitor

(optional) progress monitor to post upload completion to during the stream's close method.

monitorTask:String

(optional) task name to display during the close method.

Returns:OutputStream

a stream which accepts the new data, and transmits once closed.

Exceptions
IOException:
if encryption was enabled it could not be configured.
decryptback to summary
public InputStream decrypt(URLConnection u) throws IOException

Decrypt an input stream from get(String, String).

Parameters
u:URLConnection

connection previously created by get(String, String)}.

Returns:InputStream

stream to read plain text from.

Exceptions
IOException:
decryption could not be configured.
deleteback to summary
public void delete(String bucket, String key) throws IOException

Delete a single object.

Deletion always succeeds, even if the object does not exist.

Parameters
bucket:String

name of the bucket storing the object.

key:String

key of the object within its bucket.

Exceptions
IOException:
deletion failed due to communications error.
errorback to summary
pack-priv IOException error(final String action, final String key, final HttpURLConnection c) throws IOException
getback to summary
public URLConnection get(String bucket, String key) throws IOException

Get the content of a bucket object.

Parameters
bucket:String

name of the bucket storing the object.

key:String

key of the object within its bucket.

Returns:URLConnection

connection to stream the content of the object. The request properties of the connection may not be modified by the caller as the request parameters have already been signed.

Exceptions
IOException:
sending the request was not possible.
httpNowback to summary
private static String httpNow()
isSignedHeaderback to summary
private static boolean isSignedHeader(String name)
listback to summary
public List<String> list(String bucket, String prefix) throws IOException

List the names of keys available within a bucket.

This method is primarily meant for obtaining a "recursive directory listing" rooted under the specified bucket and prefix location. It returns the keys sorted in reverse order of LastModified time (freshest keys first).

Parameters
bucket:String

name of the bucket whose objects should be listed.

prefix:String

common prefix to filter the results by. Must not be null. Supplying the empty string will list all keys in the bucket. Supplying a non-empty string will act as though a trailing '/' appears in prefix, even if it does not.

Returns:List<String>

list of keys starting with prefix, after removing prefix (or prefix + "/")from all of them.

Exceptions
IOException:
sending the request was not possible, or the response XML document could not be parsed properly.
maxAttemptsback to summary
pack-priv IOException maxAttempts(String action, String key)
newMD5back to summary
private static MessageDigest newMD5()
openback to summary
private HttpURLConnection open(final String method, final String bucket, final String key) throws IOException
openback to summary
pack-priv HttpURLConnection open(final String method, final String bucket, final String key, final Map<String, String> args) throws IOException
propertiesback to summary
pack-priv static Properties properties(File authFile) throws FileNotFoundException, IOException
putback to summary
public void put(String bucket, String key, byte[] data) throws IOException

Atomically create or replace a single small object.

This form is only suitable for smaller contents, where the caller can reasonable fit the entire thing into memory.

End-to-end data integrity is assured by internally computing the MD5 checksum of the supplied data and transmitting the checksum along with the data itself.

Parameters
bucket:String

name of the bucket storing the object.

key:String

key of the object within its bucket.

data:byte[]

new data content for the object. Must not be null. Zero length array will create a zero length object.

Exceptions
IOException:
creation/updating failed due to communications error.
putImplback to summary
pack-priv void putImpl(final String bucket, final String key, final byte[] csum, final TemporaryBuffer buf, ProgressMonitor monitor, String monitorTask) throws IOException
removeback to summary
private static String remove(Map<String, String> m, String k)
toCleanStringback to summary
private static String toCleanString(List<String> list)
org.eclipse.jgit.transport back to summary

private final Class AmazonS3.KeyInfo

extends Object
Class Inheritance

KeyInfo enables sorting of keys by lastModified time

Field Summary

Modifier and TypeField and Description
private final long
private final String

Constructor Summary

AccessConstructor and Description
public
KeyInfo(String aname, long lsecs)

Method Summary

Modifier and TypeMethod and Description
public long
public String
Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

lastModifiedSecsback to summary
private final long lastModifiedSecs
nameback to summary
private final String name

Constructor Detail

KeyInfoback to summary
public KeyInfo(String aname, long lsecs)

Method Detail

getLastModifiedSecsback to summary
public long getLastModifiedSecs()
getNameback to summary
public String getName()
org.eclipse.jgit.transport back to summary

pack-priv Interface AmazonS3.Keys


Property names used in amazon connection configuration file.

Field Summary

Modifier and TypeField and Description
public static final String
public static final String
public static final String
public static final String
public static final String
public static final String
public static final String
public static final String
public static final String
public static final String
public static final String
public static final String

Field Detail

ACCESS_KEYback to summary
public static final String ACCESS_KEY
ACLback to summary
public static final String ACL
AWS_API_SIGNATURE_VERSIONback to summary
public static final String AWS_API_SIGNATURE_VERSION
CRYPTO_ALGback to summary
public static final String CRYPTO_ALG
CRYPTO_VERback to summary
public static final String CRYPTO_VER
DOMAINback to summary
public static final String DOMAIN
HTTP_RETRYback to summary
public static final String HTTP_RETRY
PASSWORDback to summary
public static final String PASSWORD
PROTOCOLback to summary
public static final String PROTOCOL
REGIONback to summary
public static final String REGION
SECRET_KEYback to summary
public static final String SECRET_KEY
TMP_DIRback to summary
public static final String TMP_DIR
org.eclipse.jgit.transport back to summary

private final Class AmazonS3.ListParser

extends DefaultHandler
Class Inheritance

Field Summary

Modifier and TypeField and Description
private final String
private StringBuilder
pack-priv final List<AmazonS3.KeyInfo>
private Instant
private String
private final String
pack-priv boolean

Constructor Summary

AccessConstructor and Description
pack-priv

Method Summary

Modifier and TypeMethod and Description
public void
characters(char[]
The characters.
ch
,
int
The start position in the character array.
s
,
int
The number of characters to use from the character array.
n
)

Overrides org.xml.sax.helpers.DefaultHandler.characters.

Implements org.xml.sax.ContentHandler.characters.

Receive notification of character data inside an element.

public void
endElement(final String
The Namespace URI, or the empty string if the element has no Namespace URI or if Namespace processing is not being performed.
uri
,
final String
The local name (without prefix), or the empty string if Namespace processing is not being performed.
name
,
final String
The qualified name (with prefix), or the empty string if qualified names are not available.
qName
)

Overrides org.xml.sax.helpers.DefaultHandler.endElement.

Implements org.xml.sax.ContentHandler.endElement.

Receive notification of the end of an element.

public void
ignorableWhitespace(final char[]
The whitespace characters.
ch
,
final int
The start position in the character array.
s
,
final int
The number of characters to use from the character array.
n
)

Overrides org.xml.sax.helpers.DefaultHandler.ignorableWhitespace.

Implements org.xml.sax.ContentHandler.ignorableWhitespace.

Receive notification of ignorable whitespace in element content.

pack-priv void
list()

public void
startElement(final String
The Namespace URI, or the empty string if the element has no Namespace URI or if Namespace processing is not being performed.
uri
,
final String
The local name (without prefix), or the empty string if Namespace processing is not being performed.
name
,
final String
The qualified name (with prefix), or the empty string if qualified names are not available.
qName
,
final Attributes
The attributes attached to the element. If there are no attributes, it shall be an empty Attributes object.
attributes
)

Overrides org.xml.sax.helpers.DefaultHandler.startElement.

Implements org.xml.sax.ContentHandler.startElement.

Receive notification of the start of an element.

Inherited from org.xml.sax.helpers.DefaultHandler:
endDocumentendPrefixMappingerrorfatalErrornotationDeclprocessingInstructionresolveEntitysetDocumentLocatorskippedEntitystartDocumentstartPrefixMappingunparsedEntityDeclwarning

Field Detail

bucketback to summary
private final String bucket
databack to summary
private StringBuilder data
entriesback to summary
pack-priv final List<AmazonS3.KeyInfo> entries
keyLastModifiedback to summary
private Instant keyLastModified
keyNameback to summary
private String keyName
prefixback to summary
private final String prefix
truncatedback to summary
pack-priv boolean truncated

Constructor Detail

ListParserback to summary
pack-priv ListParser(String bn, String p)

Method Detail

charactersback to summary
public void characters(char[] ch, int s, int n) throws SAXException

Overrides org.xml.sax.helpers.DefaultHandler.characters.

Implements org.xml.sax.ContentHandler.characters.

Doc from org.xml.sax.helpers.DefaultHandler.characters.

Receive notification of character data inside an element.

By default, do nothing. Application writers may override this method to take specific actions for each chunk of character data (such as adding the data to a node or buffer, or printing it to a file).

Parameters
ch:char[]

The characters.

s:int

The start position in the character array.

n:int

The number of characters to use from the character array.

Annotations
@Override
Exceptions
SAXException:
Any SAX exception, possibly wrapping another exception.
endElementback to summary
public void endElement(final String uri, final String name, final String qName) throws SAXException

Overrides org.xml.sax.helpers.DefaultHandler.endElement.

Implements org.xml.sax.ContentHandler.endElement.

Doc from org.xml.sax.helpers.DefaultHandler.endElement.

Receive notification of the end of an element.

By default, do nothing. Application writers may override this method in a subclass to take specific actions at the end of each element (such as finalising a tree node or writing output to a file).

Parameters
uri:String

The Namespace URI, or the empty string if the element has no Namespace URI or if Namespace processing is not being performed.

name:String

The local name (without prefix), or the empty string if Namespace processing is not being performed.

qName:String

The qualified name (with prefix), or the empty string if qualified names are not available.

Annotations
@Override
Exceptions
SAXException:
Any SAX exception, possibly wrapping another exception.
ignorableWhitespaceback to summary
public void ignorableWhitespace(final char[] ch, final int s, final int n) throws SAXException

Overrides org.xml.sax.helpers.DefaultHandler.ignorableWhitespace.

Implements org.xml.sax.ContentHandler.ignorableWhitespace.

Doc from org.xml.sax.helpers.DefaultHandler.ignorableWhitespace.

Receive notification of ignorable whitespace in element content.

By default, do nothing. Application writers may override this method to take specific actions for each chunk of ignorable whitespace (such as adding data to a node or buffer, or printing it to a file).

Parameters
ch:char[]

The whitespace characters.

s:int

The start position in the character array.

n:int

The number of characters to use from the character array.

Annotations
@Override
Exceptions
SAXException:
Any SAX exception, possibly wrapping another exception.
listback to summary
pack-priv void list() throws IOException
startElementback to summary
public void startElement(final String uri, final String name, final String qName, final Attributes attributes) throws SAXException

Overrides org.xml.sax.helpers.DefaultHandler.startElement.

Implements org.xml.sax.ContentHandler.startElement.

Doc from org.xml.sax.helpers.DefaultHandler.startElement.

Receive notification of the start of an element.

By default, do nothing. Application writers may override this method in a subclass to take specific actions at the start of each element (such as allocating a new tree node or writing output to a file).

Parameters
uri:String

The Namespace URI, or the empty string if the element has no Namespace URI or if Namespace processing is not being performed.

name:String

The local name (without prefix), or the empty string if Namespace processing is not being performed.

qName:String

The qualified name (with prefix), or the empty string if qualified names are not available.

attributes:Attributes

The attributes attached to the element. If there are no attributes, it shall be an empty Attributes object.

Annotations
@Override
Exceptions
SAXException:
Any SAX exception, possibly wrapping another exception.