ME-CENTRAL-1 Outage: S3 data inaccessible, cross-region copy to ap-south-1 failing

Question

Hi,

We are impacted by the ongoing ME-CENTRAL-1 outage and are attempting to migrate our S3 data to Mumbai (ap-south-1) as recommended by AWS in their service health communications, but are encountering a very high rate of errors.

We have two S3 buckets in me-central-1 (one ~175 GB for public assets, one ~270 GB for private assets). Our production site is actively showing broken images and missing assets due to S3 GET failures on objects written prior to the event.

We are attempting to migrate using:

aws s3 sync s3://source-bucket s3://target-bucket \
  --source-region me-central-1 \
  --region ap-south-1 \
  --only-show-errors

We are seeing frequent timeouts and errors. Some objects copy successfully but the majority fail. Re-running the sync helps partially but overall progress is very slow.

Questions:
1. Is there a more reliable method to copy data out of me-central-1 during an active infrastructure event like this?
2. Would S3 Batch Operations perform better than CLI sync in this scenario?
3. Is there any risk of permanent data loss for objects written prior to the event?
4. Any other recommended approaches for getting data out of a degraded region?

Note: We are on Basic Support and cannot open a technical support case, so any community guidance is appreciated. Production is down for our end users.

amazon-s3, me-central-1, disaster-recovery, outage

Jimbo Surlo · Answer

According to yesterday's 8:14AM PST update https://health.aws.amazon.com/health/status - GET error rates on S3 are elevated.  Updates are provided within your personal Health Dashboard from your console.  I'm sure that these GET error rates are affecting your transfer success.

re:Post Agent · Answer

I understand you're facing a critical situation with the ME-CENTRAL-1 outage. Here are some recommendations for your cross-region data migration:

**More Reliable Copy Methods:**

1. **Use the `cp` command with `--recursive` flag instead of `sync`** - This can sometimes handle failures more gracefully during degraded service conditions:
```
aws s3 cp s3://source-bucket/ s3://target-bucket/ \
--recursive \
--source-region me-central-1 \
--region ap-south-1
```

2. **Consider S3 Transfer Acceleration** - For large files, this can improve transfer performance by routing through AWS's optimized network paths, though availability during a regional event may vary.

3. **Batch smaller subsets** - Instead of syncing entire buckets, try copying specific prefixes or date ranges to reduce the scope of each operation and make retries more manageable.

**Regarding S3 Batch Operations:**

S3 Batch Operations can be effective for large-scale transfers under normal conditions, as they operate asynchronously in the background. However, during an active infrastructure event affecting the source region, Batch Operations would likely face the same underlying service degradation issues as CLI commands, since both depend on the health of the source region's infrastructure.

**Data Loss Concerns:**

Amazon S3 is designed with 99.999999999% (11 nines) durability. Objects written prior to an infrastructure event are stored redundantly across multiple facilities within the region. While access may be impaired during an outage, permanent data loss of successfully written objects is extremely rare. However, I cannot provide guarantees about your specific situation during an active event.

**Additional Recommendations:**

- **Monitor and retry systematically** - Keep logs of failed objects and retry those specifically rather than re-running full syncs
- **Increase timeout values** if your CLI configuration allows
- **Consider parallel operations** - Run multiple copy commands for different prefixes simultaneously to maximize throughput during windows of availability
- **Verify copied data** - Once objects successfully transfer, verify their integrity in the destination bucket

Given that you're on Basic Support and production is impacted, you may want to explore upgrading to at least Developer Support to gain access to technical support cases for guidance specific to your situation during this event.  
**Sources**  
[Troubleshoot cross-region transfer issues - AWS Transfer Family](https://docs.aws.amazon.com/transfer/latest/userguide/cross-region-transfer-issues.html)  
[Copy data from an Amazon S3 bucket to another account and Region by using the AWS CLI - AWS Prescriptive Guidance](https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/copy-data-from-an-s3-bucket-to-another-account-and-region-by-using-the-aws-cli.html)  
[Migrate between Amazon S3 storage without disrupting applications | AWS re:Post](https://repost.aws/knowledge-center/migrate-s3-without-disruption)

ME-CENTRAL-1 Outage: S3 data inaccessible, cross-region copy to ap-south-1 failing

Add your answer

Relevant content