Top Description Inners Fields Constructors Methods
org.eclipse.jgit.transport

public Class TransportAmazonS3

extends HttpTransport
implements WalkTransport
Class Inheritance
All Implemented Interfaces
org.eclipse.jgit.transport.WalkTransport
Imports
java.io.BufferedReader, .File, .FileNotFoundException, .IOException, .InputStream, .OutputStream, java.net.URLConnection, java.text.MessageFormat, java.util.ArrayList, .Collection, .Collections, .EnumSet, .HashSet, .List, .Map, .Properties, .Set, .TreeMap, org.eclipse.jgit.errors.NotSupportedException, .TransportException, org.eclipse.jgit.internal.JGitText, org.eclipse.jgit.lib.Constants, .ObjectId, .ObjectIdRef, .ProgressMonitor, .Ref, .Ref.Storage, .Repository, .SymbolicRef

Transport over the non-Git aware Amazon S3 protocol.

This transport communicates with the Amazon S3 servers (a non-free commercial hosting service that users must subscribe to). Some users may find transport to and from S3 to be a useful backup service.

The transport does not require any specialized Git support on the remote (server side) repository, as Amazon does not provide any such support. Repository files are retrieved directly through the S3 API, which uses extended HTTP/1.1 semantics. This make it possible to read or write Git data from a remote repository that is stored on S3.

Unlike the HTTP variant (see org.eclipse.jgit.transport.TransportHttp) we rely upon being able to list objects in a bucket, as the S3 API supports this function. By listing the bucket contents we can avoid relying on objects/info/packs or info/refs in the remote repository.

Concurrent pushing over this transport is not supported. Multiple concurrent push operations may cause confusion in the repository state.

See Also
WalkFetchConnection, WalkPushConnection

Nested and Inner Type Summary

Modifier and TypeClass and Description
pack-priv class

Field Summary

Modifier and TypeField and Description
pack-priv final String
bucket

Bucket the remote repository is stored in.

private final String
keyPrefix

Key prefix which all objects related to the repository start with.

pack-priv static final TransportProtocol
pack-priv final AmazonS3
s3

User information necessary to connect to S3.

pack-priv static final String
Inherited from org.eclipse.jgit.transport.HttpTransport:
connectionFactory

Constructor Summary

AccessConstructor and Description
pack-priv
TransportAmazonS3(final Repository local, final URIish uri)

Method Summary

Modifier and TypeMethod and Description
public void
close()

Implements abstract org.eclipse.jgit.transport.Transport.close.

Implements java.lang.AutoCloseable.close.

Closes this resource, relinquishing any underlying resources.

private Properties
private static Properties
public FetchConnection
openFetch()

Implements abstract org.eclipse.jgit.transport.Transport.openFetch.

Begins a new connection for fetching from the remote repository.

public PushConnection
openPush()

Implements abstract org.eclipse.jgit.transport.Transport.openPush.

Begins a new connection for pushing into the remote repository.

Inherited from org.eclipse.jgit.transport.HttpTransport:
getConnectionFactorysetConnectionFactory

Field Detail

bucketback to summary
pack-priv final String bucket

Bucket the remote repository is stored in.

keyPrefixback to summary
private final String keyPrefix

Key prefix which all objects related to the repository start with.

The prefix does not start with "/".

The prefix does not end with "/". The trailing slash is stripped during the constructor if a trailing slash was supplied in the URIish.

All files within the remote repository start with keyPrefix + "/".

PROTO_S3back to summary
pack-priv static final TransportProtocol PROTO_S3
s3back to summary
pack-priv final AmazonS3 s3

User information necessary to connect to S3.

S3_SCHEMEback to summary
pack-priv static final String S3_SCHEME

Constructor Detail

TransportAmazonS3back to summary
pack-priv TransportAmazonS3(final Repository local, final URIish uri) throws NotSupportedException

Method Detail

closeback to summary
public void close()

Implements abstract org.eclipse.jgit.transport.Transport.close.

Implements java.lang.AutoCloseable.close.

Doc from org.eclipse.jgit.transport.Transport.close.

Closes this resource, relinquishing any underlying resources. This method is invoked automatically on objects managed by the try-with-resources statement.

Close any resources used by this transport.

If the remote repository is contacted by a network socket this method must close that network socket, disconnecting the two peers. If the remote repository is actually local (same system) this method must close any open file handles used to read the "remote" repository.

AutoClosable.close() declares that it throws Exception. Implementers shouldn't throw checked exceptions. This override narrows the signature to prevent them from doing so.

Annotations
@Override
loadPropertiesback to summary
private Properties loadProperties() throws NotSupportedException
loadPropertiesFileback to summary
private static Properties loadPropertiesFile(File propsFile) throws NotSupportedException
openFetchback to summary
public FetchConnection openFetch() throws TransportException

Implements abstract org.eclipse.jgit.transport.Transport.openFetch.

Doc from org.eclipse.jgit.transport.Transport.openFetch.

Begins a new connection for fetching from the remote repository.

If the transport has no local repository, the fetch connection can only be used for reading remote refs.

Returns:FetchConnection

a fresh connection to fetch from the remote repository.

Annotations
@Override
Exceptions
TransportException:
the remote connection could not be established.
openPushback to summary
public PushConnection openPush() throws TransportException

Implements abstract org.eclipse.jgit.transport.Transport.openPush.

Doc from org.eclipse.jgit.transport.Transport.openPush.

Begins a new connection for pushing into the remote repository.

Returns:PushConnection

a fresh connection to push into the remote repository.

Annotations
@Override
Exceptions
TransportException:
the remote connection could not be established
org.eclipse.jgit.transport back to summary

pack-priv Class TransportAmazonS3.DatabaseS3

extends WalkRemoteObjectDatabase
Class Inheritance

Field Summary

Modifier and TypeField and Description
private final String
private final String
Inherited from org.eclipse.jgit.transport.WalkRemoteObjectDatabase:
INFO_PACKSINFO_REFSROOT_DIR

Constructor Summary

AccessConstructor and Description
pack-priv
DatabaseS3(final String b, final String o)

Method Summary

Modifier and TypeMethod and Description
pack-priv void
close()

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.close.

Close any resources used by this connection.

pack-priv void
deleteFile(String
name of the item to be removed, relative to the current object database.
path
)

Overrides org.eclipse.jgit.transport.WalkRemoteObjectDatabase.deleteFile.

Delete a file from the object database.

pack-priv Collection<WalkRemoteObjectDatabase>
getAlternates()

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.getAlternates.

Obtain alternate connections to alternate object databases (if any).

pack-priv Collection<String>
getPackNames()

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.getPackNames.

Obtain the list of available packs (if any).

pack-priv URIish
private Ref.Storage
pack-priv WalkRemoteObjectDatabase.FileStream
open(String
location of the file to read, relative to this objects directory (e.g. cb/95df6ab7ae9e57571511ef451cf33767c26dd2 or pack/pack-035760ab452d6eebd123add421f253ce7682355a.pack).
path
)

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.open.

Open a single file for reading.

pack-priv WalkRemoteObjectDatabase
openAlternate(String
the location of the new alternate, relative to the current object database.
location
)

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.openAlternate.

Create a new connection for a discovered alternate object database

This method is typically called by readAlternates(String) when subclasses us the generic alternate parsing logic for their implementation of getAlternates().

pack-priv Map<String, Ref>
private void
private Ref
private String
resolveKey(String subpath)

pack-priv OutputStream
writeFile(final String
name of the file to write, relative to the current object database.
path
,
final ProgressMonitor
(optional) progress monitor to post write completion to during the stream's close method.
monitor
,
final String
(optional) task name to display during the close method.
monitorTask
)

Overrides org.eclipse.jgit.transport.WalkRemoteObjectDatabase.writeFile.

Open a remote file for writing.

pack-priv void
writeFile(String
name of the file to write, relative to the current object database.
path
,
byte[]
complete new content of the file.
data
)

Overrides org.eclipse.jgit.transport.WalkRemoteObjectDatabase.writeFile.

Atomically write a remote file.

Inherited from org.eclipse.jgit.transport.WalkRemoteObjectDatabase:
deleteRefdeleteRefLogopenReaderreadAlternatesreadPackedRefswriteInfoPackswriteRef

Field Detail

bucketNameback to summary
private final String bucketName
objectsKeyback to summary
private final String objectsKey

Constructor Detail

DatabaseS3back to summary
pack-priv DatabaseS3(final String b, final String o)

Method Detail

closeback to summary
pack-priv void close()

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.close.

Doc from org.eclipse.jgit.transport.WalkRemoteObjectDatabase.close.

Close any resources used by this connection.

If the remote repository is contacted by a network socket this method must close that network socket, disconnecting the two peers. If the remote repository is actually local (same system) this method must close any open file handles used to read the "remote" repository.

Annotations
@Override
deleteFileback to summary
pack-priv void deleteFile(String path) throws IOException

Overrides org.eclipse.jgit.transport.WalkRemoteObjectDatabase.deleteFile.

Doc from org.eclipse.jgit.transport.WalkRemoteObjectDatabase.deleteFile.

Delete a file from the object database.

Path may start with ../ to request deletion of a file that resides in the repository itself.

When possible empty directories must be removed, up to but not including the current object database directory itself.

This method does not support deletion of directories.

Parameters
path:String

name of the item to be removed, relative to the current object database.

Annotations
@Override
Exceptions
IOException:
deletion is not supported, or deletion failed.
getAlternatesback to summary
pack-priv Collection<WalkRemoteObjectDatabase> getAlternates() throws IOException

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.getAlternates.

Doc from org.eclipse.jgit.transport.WalkRemoteObjectDatabase.getAlternates.

Obtain alternate connections to alternate object databases (if any).

Alternates are typically read from the file org.eclipse.jgit.lib.Constants#INFO_ALTERNATES or org.eclipse.jgit.lib.Constants#INFO_HTTP_ALTERNATES. The content of each line must be resolved by the implementation and a new database reference should be returned to represent the additional location.

Alternates may reuse the same network connection handle, however the fetch connection will close() each created alternate.

Returns:Collection<WalkRemoteObjectDatabase>

list of additional object databases the caller could fetch from; null or empty list if none are configured.

Annotations
@Override
Exceptions
IOException:
The connection is unable to read the remote repository's list of configured alternates.
getPackNamesback to summary
pack-priv Collection<String> getPackNames() throws IOException

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.getPackNames.

Doc from org.eclipse.jgit.transport.WalkRemoteObjectDatabase.getPackNames.

Obtain the list of available packs (if any).

Pack names should be the file name in the packs directory, that is pack-035760ab452d6eebd123add421f253ce7682355a.pack. Index names should not be included in the returned collection.

Returns:Collection<String>

list of pack names; null or empty list if none are available.

Annotations
@Override
Exceptions
IOException:
The connection is unable to read the remote repository's list of available pack files.
getURIback to summary
pack-priv URIish getURI()

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.getURI.

Annotations
@Override
looseback to summary
private Ref.Storage loose(Ref r)
openback to summary
pack-priv WalkRemoteObjectDatabase.FileStream open(String path) throws IOException

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.open.

Doc from org.eclipse.jgit.transport.WalkRemoteObjectDatabase.open.

Open a single file for reading.

Implementors should make every attempt possible to ensure FileNotFoundException is used when the remote object does not exist. However when fetching over HTTP some misconfigured servers may generate a 200 OK status message (rather than a 404 Not Found) with an HTML formatted message explaining the requested resource does not exist. Callers such as WalkFetchConnection are prepared to handle this by validating the content received, and assuming content that fails to match its hash is an incorrectly phrased FileNotFoundException.

This method is recommended for already compressed files like loose objects and pack files. For text files, see openReader(String).

Parameters
path:String

location of the file to read, relative to this objects directory (e.g. cb/95df6ab7ae9e57571511ef451cf33767c26dd2 or pack/pack-035760ab452d6eebd123add421f253ce7682355a.pack).

Returns:WalkRemoteObjectDatabase.FileStream

a stream to read from the file. Never null.

Annotations
@Override
Exceptions
IOException:
The connection is unable to read the remote's file, and the failure occurred prior to being able to determine if the file exists, or after it was determined to exist but before the stream could be created.
openAlternateback to summary
pack-priv WalkRemoteObjectDatabase openAlternate(String location) throws IOException

Implements abstract org.eclipse.jgit.transport.WalkRemoteObjectDatabase.openAlternate.

Doc from org.eclipse.jgit.transport.WalkRemoteObjectDatabase.openAlternate.

Create a new connection for a discovered alternate object database

This method is typically called by readAlternates(String) when subclasses us the generic alternate parsing logic for their implementation of getAlternates().

Parameters
location:String

the location of the new alternate, relative to the current object database.

Returns:WalkRemoteObjectDatabase

a new database connection that can read from the specified alternate.

Annotations
@Override
Exceptions
IOException:
The database connection cannot be established with the alternate, such as if the alternate location does not actually exist and the connection's constructor attempts to verify that.
readAdvertisedRefsback to summary
pack-priv Map<String, Ref> readAdvertisedRefs() throws TransportException
readLooseRefsback to summary
private void readLooseRefs(TreeMap<String, Ref> avail) throws TransportException
readRefback to summary
private Ref readRef(TreeMap<String, Ref> avail, String rn) throws TransportException
resolveKeyback to summary
private String resolveKey(String subpath)
writeFileback to summary
pack-priv OutputStream writeFile(final String path, final ProgressMonitor monitor, final String monitorTask) throws IOException

Overrides org.eclipse.jgit.transport.WalkRemoteObjectDatabase.writeFile.

Doc from org.eclipse.jgit.transport.WalkRemoteObjectDatabase.writeFile.

Open a remote file for writing.

Path may start with ../ to request writing of a file that resides in the repository itself.

The requested path may or may not exist. If the path already exists as a file the file should be truncated and completely replaced.

This method creates any missing parent directories, if necessary.

Parameters
path:String

name of the file to write, relative to the current object database.

monitor:ProgressMonitor

(optional) progress monitor to post write completion to during the stream's close method.

monitorTask:String

(optional) task name to display during the close method.

Returns:OutputStream

stream to write into this file. Caller must close the stream to complete the write request. The stream is not buffered and each write may cause a network request/response so callers should buffer to smooth out small writes.

Annotations
@Override
Exceptions
IOException:
writing is not supported, or attempting to write the file failed, possibly due to permissions or remote disk full, etc.
writeFileback to summary
pack-priv void writeFile(String path, byte[] data) throws IOException

Overrides org.eclipse.jgit.transport.WalkRemoteObjectDatabase.writeFile.

Doc from org.eclipse.jgit.transport.WalkRemoteObjectDatabase.writeFile.

Atomically write a remote file.

This method attempts to perform as atomic of an update as it can, reducing (or eliminating) the time that clients might be able to see partial file content. This method is not suitable for very large transfers as the complete content must be passed as an argument.

Path may start with ../ to request writing of a file that resides in the repository itself.

The requested path may or may not exist. If the path already exists as a file the file should be truncated and completely replaced.

This method creates any missing parent directories, if necessary.

Parameters
path:String

name of the file to write, relative to the current object database.

data:byte[]

complete new content of the file.

Annotations
@Override
Exceptions
IOException:
writing is not supported, or attempting to write the file failed, possibly due to permissions or remote disk full, etc.