aws py cli

Prerequisites

This has been tested on Ubuntu 12.04 and 14.04, but may also work on others.

Steps

Installation

Install the tools with the following commands:

sudo apt-get install python-pip -y
sudo pip install awscli

Personal Configuration

If you just want to use this for a single account with manual commands (e.g. not scripts), then perform the following steps. If you want to configure this for multiple accounts and/or scripts, then it is probably best to run the “Setup For Scripts” section instead.

Run the following command and answer the questions

aws configure

When it asks for the default output format, your choices are

json

,

table

, or

text

. Personally, I use

json

Setup For Scripts

If you followed the previous section, then you do not need to run this one.

Now we need to set up our AWS credentials for automatic authentication

mkdir $HOME/.aws
vim $HOME/.aws/config

File contents

[default]
aws_access_key_id = [ID HERE]
aws_secret_access_key = [KEY HERE]
region = eu-west-1
It won’t work if you put quotation marks around the values.

The configuration needs to be there, otherwise you need to set the AWS_CONFIG_FILE environment variable to its location e.g.

export AWS_CONFIG_FILE="`echo $HOME`/.aws/config"

Protect the file from other users who have access to the same machine.

chmod 600 $HOME/.aws/config

Testing

You can test it’s working with a simple command t fetch the regions from AWS:

aws ec2 describe-regions

Now you can use the CLI to transfer files to and from s3 like so:

aws s3 cp /path/to/local/file.txt s3://my-bucket/sub-folder/file.txt

or the other way around

aws s3 cp s3://my-bucket/sub-folder/file.txt /path/to/local/file.txt
The S3 file movement commands are pretty much the same as with linux so it should feel natural. e.g. cp, mv and the –recursive switch etc

References

 

AWS CLI S3 Cheatsheet

It’s much faster to interface with S3 files through the CLI than it is to use the web browser. Here is a cheatsheet of useful S3 based CLI commands.

Remove All Files In A Bucket

aws s3 rm --recursive s3://my-bucket

One cannot use the

*

character as a wildcard. S3 considers this a valid character for a filename. Hence the use of the

–recursive

flag.

Copy A Bucket

aws s3 cp --recursive s3://my-bucket s3://my-second-bucket
One can use this to copy files between two different accounts, as long as one of the account’s buckets is publicly accessable!
The files go direct from server to server (e.g. not through you) so this is super fast!

Move A Bucket’s Contents

Same as before but with “mv” instead of “cp”

Upload To Bucket

aws s3 cp /path/to/local/file.txt s3://my-bucket/sub-folder/file.txt

Download File From Bucket

aws s3 cp s3://my-bucket/sub-folder/file.txt /path/to/local/file.txt

assyrian technical blog