Peculiar behaviour on buckets in ap-southeast-4 (Melbourne) region

0

API keys used have full AdministratorAccess privileges, and the Melbourne region is enabled for the account. For historical reasons the account's "home" region, as set in ~/.aws/config is Sydney.

$ aws configure get region
ap-southeast-2
$ 

Create some buckets all around the world, don't worry about encryption or policies or ACLs or anything like that:

$ aws s3api create-bucket --bucket aaaa-monday-bucket-sydney --create-bucket-configuration LocationConstraint=ap-southeast-2
{
    "Location": "http://aaaa-monday-bucket-sydney.s3.amazonaws.com/"
}
$ aws s3api create-bucket --bucket aaaa-monday-bucket-melbourne --region ap-southeast-4 --create-bucket-configuration LocationConstraint=ap-southeast-4
{
    "Location": "http://aaaa-monday-bucket-melbourne.s3.amazonaws.com/"
}
$ aws s3api create-bucket --bucket aaaa-monday-bucket-tokyo --region ap-northeast-1 --create-bucket-configuration LocationConstraint=ap-northeast-1
{
    "Location": "http://aaaa-monday-bucket-tokyo.s3.amazonaws.com/"
}
$ aws s3api create-bucket --bucket aaaa-monday-bucket-london --region eu-west-2 --create-bucket-configuration LocationConstraint=eu-west-2
{
    "Location": "http://aaaa-monday-bucket-london.s3.amazonaws.com/"
}
$ aws s3api create-bucket --bucket aaaa-monday-bucket-saopaulo --region sa-east-1 --create-bucket-configuration LocationConstraint=sa-east-1
{
    "Location": "http://aaaa-monday-bucket-saopaulo.s3.amazonaws.com/"
}
$ aws s3api create-bucket --bucket aaaa-monday-bucket-singapore --region ap-southeast-1 --create-bucket-configuration LocationConstraint=ap-southeast-1
{
    "Location": "http://aaaa-monday-bucket-singapore.s3.amazonaws.com/"
}
$ aws s3api create-bucket --bucket aaaa-monday-bucket-canada --region ca-central-1 --create-bucket-configuration LocationConstraint=ca-central-1
{
    "Location": "http://aaaa-monday-bucket-canada.s3.amazonaws.com/"
}
$ aws s3api create-bucket --bucket aaaa-monday-bucket-ohio --region us-east-2 --create-bucket-configuration LocationConstraint=us-east-2
{
    "Location": "http://aaaa-monday-bucket-ohio.s3.amazonaws.com/"
}
$ aws s3 ls
2023-04-17 10:55:15 aaaa-monday-bucket-canada
2023-04-17 10:42:13 aaaa-monday-bucket-london
2023-04-17 10:39:32 aaaa-monday-bucket-melbourne
2023-04-17 10:56:41 aaaa-monday-bucket-ohio
2023-04-17 10:42:44 aaaa-monday-bucket-saopaulo
2023-04-17 10:53:19 aaaa-monday-bucket-singapore
2023-04-17 10:39:05 aaaa-monday-bucket-sydney
2023-04-17 10:41:38 aaaa-monday-bucket-tokyo
.
.
.
$

Upload an object to each of the buckets:

$ echo test > test.txt
$ for i in canada london melbourne ohio saopaulo singapore sydney tokyo; do echo $i; echo "==="; aws s3 cp test.txt s3://aaaa-monday-bucket-$i ; echo; done
canada
===
upload: ./test.txt to s3://aaaa-monday-bucket-canada/test.txt

london
===
upload: ./test.txt to s3://aaaa-monday-bucket-london/test.txt

melbourne
===
upload failed: ./test.txt to s3://aaaa-monday-bucket-melbourne/test.txt An error occurred (IllegalLocationConstraintException) when calling the PutObject operation: The ap-southeast-4 location constraint is incompatible for the region specific endpoint this request was sent to.

ohio
===
upload: ./test.txt to s3://aaaa-monday-bucket-ohio/test.txt  

saopaulo
===
upload: ./test.txt to s3://aaaa-monday-bucket-saopaulo/test.txt

singapore
===
upload: ./test.txt to s3://aaaa-monday-bucket-singapore/test.txt

sydney
===
upload: ./test.txt to s3://aaaa-monday-bucket-sydney/test.txt  

tokyo
===
upload: ./test.txt to s3://aaaa-monday-bucket-tokyo/test.txt  

$ 

Why does Melbourne fail?

It works when the region is added as a flag, but why the need to do this for Melbourne and not anywhere else?

$ aws s3 cp test.txt s3://aaaa-monday-bucket-melbourne --region ap-southeast-4
upload: ./test.txt to s3://aaaa-monday-bucket-melbourne/test.txt
$ 
profile picture
EXPERT
Steve_M
asked a year ago264 views
2 Answers
0
Accepted Answer

I tried running it with the "--debug" option in my environment.
For reasons unknown, they were trying to send it toward S3 in the default region.
For example, if the default region is "ap-northeast-1", the endpoint would be something like "https://aaaa-monday-bucket-melbourne.s3.ap-northeast-1.amazonaws.com:443".
The above may be a bug.

profile picture
EXPERT
answered a year ago
  • Thanks for this, and good thinking, but that happens in all regions, e.g. "host:aaaa-monday-bucket-london.s3.ap-southeast-2.amazonaws.com" is the endpoint in my debug output for the eu-west-2 bucket, but the "aws s3 cp" command still works.

    I may just put it down to the Melbourne region not being 100% finished yet ....

  • The "ap-southeast-4" region is a region created after 2019, so it seems that an error will occur if you do not specify the region and make a request. https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html

    For S3 buckets in Regions launched after March 20, 2019, the DNS server doesn't route your request directly to the AWS Region where your bucket resides. It returns an HTTP 400 Bad Request error instead. For more information, see Making requests.

  • I think I see what you mean.

    For a bucket in an old region, e.g. Sydney 2012, aws s3 cp test.txt s3://my_bucket/ would expand the bucket URI my_bucket.s3.amazonaws.com and this would be sent to us-east-1 (because no region is specified), and from here AWS S3 would redirect to Sydney.

    For a bucket in a new region, e.g. Melbourne 2023, the same command would expand the bucket URI the same, and send it to us-east-1 where it would error (because it won't redirect for buckets in regions launched after 2019).

    Change the command to aws s3 cp test.txt s3://my_bucket/ --region ap-southeast-4 and the bucket URI would be expanded to my_bucket.s3.ap-southeast-4.amazonaws.com and this would be sent direct to Melbourne, which will be successful.

    A year later I finally have an answer! Thanks @Riku_Kobayashi

0

What does aws s3api get-bucket-location --bucket aaaa-monday-bucket-melbourne return? I see under aws s3api create-bucket help that ap-southeast-4 isn't listed in the LocationContraint syntax. Maybe the doco is out of date, or maybe it really doesn't work?

EXPERT
answered a year ago
  • Thanks, here's the get-bucket-location:

    $ aws s3api get-bucket-location --bucket aaaa-monday-bucket-melbourne { "LocationConstraint": "ap-southeast-4" }

    Your mention of the man page got me thinking that maybe my version of the AWS CLI is out-dated (the Melbourne region came onstream earlier this year), so I updated to the latest "aws-cli/2.11.13 Python/3.11.3 Linux/6.2.10-200.fc37.x86_64". But that section of "aws s3api create-bucket help" is still the same, and still no joy with a plain "aws s3 cp" without specifying the region.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions