NAME¶

s3ls - List S3 buckets and bucket contents

SYNOPSIS¶

s3ls [options]

s3ls [options] [ [ bucket | bucket/item ] ...]

 Options:
   --access-key    AWS Access Key ID
   --secret-key    AWS Secret Access Key
   --long
 Environment:
   AWS_ACCESS_KEY_ID
   AWS_ACCESS_KEY_SECRET

OPTIONS¶

--help

Print a brief help message and exits.

--man

Prints the manual page and exits.

--verbose

Output what is being done as it is done.

--access-key and --secret-key

Specify the "AWS Access Key Identifiers" for the AWS account. --access-key is the "Access Key ID", and --secret-key is the "Secret Access Key". These are effectively the "username" and "password" to the AWS account, and should be kept confidential.

The access keys MUST be specified, either via these command line parameters, or via the AWS_ACCESS_KEY_ID and AWS_ACCESS_KEY_SECRET environment variables.

Specifying them on the command line overrides the environment variables.

--secure

Uses SSL/TLS HTTPS to communicate with the AWS service, instead of HTTP.

--long

ENVIRONMENT VARIABLES¶

AWS_ACCESS_KEY_ID and AWS_ACCESS_KEY_SECRET

Specify the "AWS Access Key Identifiers" for the AWS account. AWS_ACCESS_KEY_ID contains the "Access Key ID", and AWS_ACCESS_KEY_SECRET contains the "Secret Access Key". These are effectively the "username" and "password" to the AWS service, and should be kept confidential.

The access keys MUST be specified, either via these environment variables, or via the --access-key and --secret-key command line parameters.

If the command line parameters are set, they override these environment variables.

CONFIGURATION FILE¶

The configuration options will be read from the file "~/.s3-tools" if it exists. The format is the same as the command line options with one option per line. For example, the file could contain:

    --access-key <AWS access key>
    --secret-key <AWS secret key>
    --secure

This example configuration file would specify the AWS access keys and that a secure connection using HTTPS should be used for all communications.

DESCRIPTION¶

Lists the buckets owned by the user, or all the item keys in a given bucket, or attributes associated with a given item.

If no buckets or bucket/itemkey is specified on the command line, all the buckets owned by the user are listed.

If the "--long" option is specified, the creation date of each bucket is also output.

If a bucket name is specified on the command line, all the item keys in that bucket are listed.

If the "--long" option is specified, the ID and display string of the item owner, the creation date, the MD5, and the size of the item are also output.

If a bucket name and an item key, separated by a slash character, is specified on the command line, then the bucket name and the item key are output. This is useful to check that the item actually exists.

If the "--long" option is specified, all the HTTP attributes of the item are also output. This will include Content-Length, Content-Type, ETag (which is the MD5 of the item contents), and Last-Modifed.

It may also include the HTTP attributes Content-Language, Expires, Cache-Control, Content-Disposition, and Content-Encoding.

It will also include any x-amz- metadata headers.

BUGS¶

Report bugs to Mark Atwood mark@fallenpegasus.com.

Occasionally the S3 service will randomly fail for no externally apparent reason. When that happens, this tool should retry, with a delay and a backoff.

Access to the S3 service can be authenticated with a X.509 certificate, instead of via the "AWS Access Key Identifiers". This tool should support that.

It might be useful to be able to specify the "AWS Access Key Identifiers" in the user's "~/.netrc" file. This tool should support that.

Errors and warnings are very "Perl-ish", and can be confusing.

Trying to access a bucket or item that does not exist or is not accessible by the user generates less than helpful error messages.

This tool does not efficiently handle listing huge buckets, as it downloads and parses the entire bucket listing, before it outputs anything.

This tool does not take advantage of the prefix, delimiter, and hierarchy features of the AWS S3 key listing API.

AUTHOR¶

Written by Mark Atwood mark@fallenpegasus.com.

Many thanks to Wotan LLC <http://wotanllc.com>, for supporting the development of these S3 tools.

Many thanks to the Amazon AWS engineers for developing S3.

SEE ALSO¶

These tools use the Net::Amazon:S3 Perl module.

The Amazon Simple Storage Service (S3) is documented at <http://aws.amazon.com/s3>.