S3 Bucket Tagging with boto3 works in some regions but not others?

0

Does anyone know if there are limitations with boto3 working in some regions but not others?

I'm tagging S3 buckets with a python script using boto3. I'm running the script in us-east-1 region, and I've tried running it in several other regions. The script runs fine, except when processing buckets in regions like ap-east-1, af-south-1, me-south-1. For that sake of clarity, I'll call these regions the "outcast" regions. It lists the buckets but does not perform any action, such as printing a notice that "bucket XYZ already has the tag", or "tagging bucket XYZ with tag <some key>: <some value>". It tags buckets with no problems in ap-southeast-2, us-xxxxx-xx, eu-xxxx-xx, and ca-central-1 regions. I've checked my account access to these regions, and I have access to all of the regions including the "outcast" regions. The script lists all the buckets and iterates through the list. It skips buckets that are already tagged and tags buckets that are not tagged, but when it gets to any of the buckets in the outcast regions, it prints the message "Processing Bucket XYZ" and then simply moves on to the next bucket in the list without reporting whether or not that bucket has the tags. I've run the script in some of outcast regions as a test and saw the same behavior there. I've also noticed that the AWS Resource Groups and Tagging Editor does not list buckets in the "Outcast" regions.

  • Hi Jeff.

    Can you confirm the bucket policies for buckets in those regions allow access/tagging to your credentials?

  • Thanks for suggestion Jose. I'll check the policies.

  • I found one of the buckets was created by one of our teams as a "test" bucket and had a very restrictive policy applied to it. I'll take a look my boto3 S3 error handling to see why it didn't surface the "access denied" error. Thanks again!

Jeff L
asked 10 months ago361 views
2 Answers
0
Accepted Answer

Hi,

I would suggest that you analyze the CloudTail activity logs in the outcast regions to see the exact requests sent to S3 in those regions. When you compare them with same logs in working regions, you may find the root cause of your problem.

such CloudTrail analysis has helped me in similar situations where my code was behaving differently in different regions.

Didier

profile pictureAWS
EXPERT
answered 10 months ago
  • Thank you Didier. I'll check CloudTrail to see if I can find anything there.

0

UPDATE - I was able to pinpoint an error message in CloudTrail for one of the buckets that my script just kind of passed over, but I wasn't able to find the exact error in CloudTrail for other buckets in the "outcast" regions. I was however able to coax an error message on those buckets using the AWS cli. When performing a simple "get-bucket-policy" request on a bucket in me-south-1, the AWS cli returned the following error: "An error occurred (IllegalLocationConstraintException) when calling the GetBucketPolicy operation: The me-south-1 location constraint is incompatible for the region specific endpoint this request was sent to." I do use the shell environment variable "AWS_DEFAULT_REGION" which I change depending on which AWS region I am running the script in, but because S3 buckets are considered a "Global" resource, a default region "shouldn't" really matter. But apparently it does in some regions. When re-running the AWS cli "get-bucket-policy" with the --region option set to me-south-1, the command returned the data with no problems. So, lesson learned... some of the AWS regions do indeed require API calls to have the region specified in the call.

Jeff L
answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions