boto3 upload to s3 not giving errors, but files do not show up in bucket


Just as the title says. Initially, I attempted using the sample code given from the docs linked below, with the exception that I passed my access key and secret key when creating the client object.

I was using driver code that printed the file name and 'success' when the function returned true. All files were being printed with 'success.' But when I checked my bucket, no uploads appeared.

On the off chance it was something wrong in the function, I changed my code to just use

 response = s3_client.upload_file(file_name, bucket, file_name)

directly in the driver and without the rest of the function, still the code ran without errors, but no uploads appeared in my bucket.

In the shell I logged in with the client and was able to successfully produce a list of my buckets, so I know it's connecting and the access keys are working in that sense.

All I can think is maybe I'm missing some formatting thing with my bucket names? I'm just using the plain text name as a string, for example 'mybucket1234.' Do I need to give it the full url?

Edited by: lwaxana on Feb 16, 2020 7:46 AM

asked 3 years ago110 views
1 Answer

I was having the same problem with code adapted from

s3_client.upload_file completed without an exception but the file was not visible in the S3 web console or via "aws s3 ls s3://bucket-name"

The issue was with these lines:

    if object_name is None:
        object_name = file_name

file_name is actually a file path and the upload seems to silently fail if it contains path separators. The workaround is either to always provide an object_name or change it to:

    if object_name is None:
        object_name = file_name.split(os.path.sep)[-1:][0]
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions