Backing up an s3 bucket to another s3 bucket

In a project I am working on in Rumble Labs we had cause to want to backup an entire s3 bucket before making potentially destructive changes.

After some research I found that AWS have an officially supported client that would allow you to do just this. In this guide I will walk you through how to install the AWS client, create a new bucket and sync your stuff across to this new bucket.

## Installing and Configuring AWS CLI

I had both Python and Pip installed so was able to install everything using Pip with the following command:

sudo pip install awscli

If you don’t have Pip installed you can follow this guide.

Now we need to configure our AWS client by giving it our keys and such, this is explained in full in the official guide but all I needed to do was run the following command

aws configure

You will be asked a series of questions;

• Access Key (required)
• Secret Key (required)
• Default Region (I left this blank)
• Default Output format (I left this blank)

You can run the configure command as many times as you like, any blank answers will assume the previous setting is staying unchanged.

## Backing up your s3 Bucket

The first thing we want to do is create a new bucket to house our backup

aws s3 mb s3://my-backup-bucket

What this does is tell aws that we are performing an s3 related action, the action we are performing is mb (make bucket) and we are supplying a bucket name, in this case my-backup-bucket.

Now we have everything in place to copy our stuff into our new bucket, we do this with the aws sync command:

aws s3 sync s3://my-current-bucket s3://my-backup-bucket

What this does is tell aws again that we are performing an s3 action, this time we are performing the sync action. This action takes two properties, the bucket we are copying from and the bucket we are copying to.

Once you run this you will receive the output of the files copying in your terminal, which I wanted because I like to keep an eye on these things. If you don’t want to see this I see there is a --quiet flag listed in the documentation for sync.