How do I replicate existing objects in Amazon S3?

Short description

Use Batch Replication in Amazon S3 to replicate objects that existed before a replication configuration was in place. You can also use Batch Replication to replicate previously replicated objects and objects that failed replication.

To replicate existing objects using Amazon S3 Batch Replication, complete the following tasks:

• Configure the AWS Identity and Access Management (IAM) S3 replication role and S3 batch operation role. The S3 replication role performs the replication of the objects. The S3 batch role configures replication requirements.
• Create a Batch Replication job
• (Optional) Review the Completion report for issues

Prerequisites

To assist you in the replication process, complete the following tasks:

• If you have fewer than 10 million objects in your S3 bucket, then use the sync command to copy objects to a bucket. For more information, see How can I copy all objects from one Amazon S3 bucket to another bucket?
• The replication configuration for the source bucket must already exist.
• The replication role must have permissions to replicate the Amazon S3 action on both the source and destination buckets.

For more information, see S3 Batch Replication considerations.

Resolution

Configure S3 replication and Batch operation roles

Create an IAM role that grants Amazon S3 permissions to replicate on your behalf. Then, attach a Batch Replication policy to the batch operation role. For more information, see Setting up permissions and Configuring IAM policies for Batch Replication.

Create a Batch Replication job

Before you create a Batch Replication job, confirm that you're replicating the objects for the first time to a destination. Or, that you're running the Batch job on existing S3 replication rules. For a first replication to a destination, see Create a Batch Replication job for a first replication rule or new destination. For existing replication rules, see Create a Batch Replication job for existing replication rules.

(Optional) Review the Completion report for issues

After you run a Batch Replication job, you receive a completion report that displays the status of the job. To troubleshoot issues, review and filter this report for either Failed tasks only or All tasks. For information on why a Batch Replication job failed, see Amazon S3 replication failure reasons.

Related information

Replicating existing objects with S3 Batch Replication