AWS S3 multipart upload hangs up

0

I suddenly cannot upload files more than 2mb by AWS S3 Console and by the CLI. This issue started a few days ago. I failed to upload files of 3mb and 8mb. The file of 2mb has been uploaded succesfully. In AWS Console the process is just hangs up and in CLI I receive an error:

> aws s3 cp ./app/static/openai/spam.jsonl s3://openai-all-data 
upload failed: app/static/openai/spam.jsonl to s3://openai-all-data/spam.jsonl Connection was closed before we received a valid response from endpoint URL: "https://openai-all-data.s3.us-east-1.amazonaws.com/spam.jsonl?uploadId=UK.4REWj9ttY80MRH2SE3Nt2sxHj.AEikZoGF0x2NHHblY1rEsy6k3OTWesJ5i4.EPgi5Bm7K31XuunjHQlY2oIlrD_lI11T1Co6C5n.8xXpwS0mqDpQZp_lCXuDJRv9&partNumber=1".

My internet upload speed is OK and I can upload large files to Google Drive.

MarkBB
asked 6 months ago274 views
2 Answers
0

Hi,

Can you try to extend connect timeout with --cli-connect-timeout 6000

See for details: https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html

Also you may want to specify explicitly the region where your bucket was created with --region

Best,

Didier

profile pictureAWS
EXPERT
answered 6 months ago
0

It started with:

aws s3 cp ./app/static/openai/spam.jsonl s3://openai-all-data --region us-east-1 --cli-connect-timeout 6000
Completed 8.2 MiB/29.8 MiB (5.3 MiB/s) with 1 file(s) remaining   
Completed 16.8 MiB/29.8 MiB (147.2 KiB/s) with 1 file(s) remaining
Completed 21.2 MiB/29.8 MiB (3.6 KiB/s) with 1 file(s) remaining

and after that it stuck completely.

MarkBB
answered 6 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions