- Newest
- Most votes
- Most comments
Hello.
As stated in the following documentation, S3 allows the use of a plus sign in object names, but it may require additional code processing and may need to be URL-encoded or referenced in hex.
Therefore, I recommend replacing the plus sign with another character.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html
The following characters in a key name might require additional code handling and most likely must be URL encoded or referenced as HEX. Some of these characters are non-printable characters that your browser might not handle, which also require special handling:
- Ampersand (&)
- Dollar sign ($)
- ASCII character ranges 00–1F hex (0–31 decimal) and 7F (127 decimal)
- At symbol (@)
- Equal sign (=)
- Semicolon (;)
- Forward slash (/)
- Colon (:)
- Plus sign (+)
- Space – Significant sequences of spaces might be lost in some cases (especially multiple spaces)
- Comma (,)
- Question mark (?)
Alternatively, you will need to take measures such as encoding special characters as shared in the URL below.
https://repost.aws/questions/QU8t5jKaakToafeUdKvvr_1Q/s3-batch-job-error-task-target-couldn-t-be-url-decoded#ANPhrM0-0rSnKT7uxAPSUw8g
It isn't a bug. This documentation section https://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-ops-create-job.html#create-manifest-file says:
The contents of the manifest must be URL-encoded.
The + character is a reserved character in URL encoding rules, and it must be replaced with the URL-encoded sequence %2B. There are more details on the rules in the Wikipedia article: https://en.wikipedia.org/wiki/Percent-encoding#Reserved_characters
This appears to be a specific issue with how S3 Batch Operations handles the plus sign (+) in object keys. The error message you're receiving suggests an authorization issue, but since you've confirmed the role has the proper ListBucket permissions and you can successfully copy these same objects using the CLI, it's likely related to how S3 Batch Operations processes special characters in object keys.
S3 Batch Operations has known limitations with certain special characters in object keys. While your specific issue with the plus sign isn't explicitly documented in the sources available, it's similar to a known limitation where S3 Batch Operations doesn't support object keys containing line feed (\n) or carriage return (\r) characters, even when they're URL-encoded.
For objects with special characters like the plus sign in their keys, you might need to use alternative approaches:
- Use AWS DataSync for large-scale transfers between S3 buckets
- Create a custom solution using the AWS SDK to handle these specific objects
- Replace the special characters in object keys before running the batch operation
When planning batch operations for production environments with large amounts of data, it's advisable to run tests on a representative sample of objects first and review the completion report to identify any failed objects and their specific error patterns.
Sources
Error in S3 batch operation job | AWS re:Post
Use S3 Batch Operations to copy large objects | AWS re:Post
Relevant content
- asked 4 months ago
- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated 9 months ago
Since this AI answer essentially telling me "Don't use plus signs" I don't think I should accept it. I'm hoping someone can confirm the findings and either fix the issue, or at least this will be a record for future people who run into the same issue.