Error in S3 batch operation job

This error occurs because S3 Batch Operations currently doesn't support object keys containing line feed (\n) or carriage return (\r) characters, even when they're URL-encoded.

To resolve this issue, you can:

1. Replace special characters in object keys with their XML entity codes and ensure that object keys meet the Amazon S3 naming conventions.

2. Ensure all object keys are properly URL encoded, though note that even with URL encoding, line feed and carriage return characters specifically are not supported by S3 Batch Operations.

3. Break your job into smaller jobs, focusing on objects without problematic characters in their keys.

For the objects that failed to copy, you might need to use an alternative approach:

• Use AWS DataSync, which is designed for large-scale data transfers between S3 buckets, including cross-account transfers.

• Use a custom solution with the AWS SDK to handle these specific objects with problematic keys.

To ensure all objects are copied before applying to your production environment with TBs of data, review the completion report from your batch job to identify all failed objects. The completion report will provide details about which specific objects failed and why, allowing you to address them separately.

Amazon S3 imposes a task-failure threshold on Batch Operations jobs - if more than 50% of tasks fail after running at least 1,000 tasks, the entire job will fail. Keep this in mind when planning your production migration.
Sources
Troubleshooting Batch Operations - Amazon Simple Storage Service
Data copy using s3 batch | AWS re:Post
Tracking job status and completion reports - Amazon Simple Storage Service