How do we select the entire contents of a bucket (all images), and bulk copy URL list for all the objects?

0

How do we select the entire contents of a bucket, and bulk copy URL list for all the objects? Or generate a CSV with all of the URL's for every object in the bucket?

asked 2 years ago879 views
2 Answers
0

You can't do it with S3 features alone.

But you can get a list of objects using S3's advanced API.
list-objects-v2 — AWS CLI 2.5.8 Command Reference

Then, you can format the acquired list into an easy-to-use form using another module such as jq.

$ aws s3api list-objects-v2 --bucket bucketname | jq -r '.Contents[] | ["s3://bucketname/" + .Key] | @csv'
"s3://bucketname/html/API_AddTags.md.html"
"s3://bucketname/html/API_AlgorithmSpecification.md.html"
"s3://bucketname/html/API_AlgorithmStatusDetails.md.html"
"s3://bucketname/html/API_AlgorithmStatusItem.md.html"
"s3://bucketname/html/API_AlgorithmSummary.md.html"
"s3://bucketname/html/API_AlgorithmValidationProfile.md.html"
"s3://bucketname/html/API_AlgorithmValidationSpecification.md.html"
:
profile picture
EXPERT
iwasa
answered 2 years ago
0

yes you can do it by using this command this will create a csv file with two column first is file name and the second is Object _url

*** **for /f "tokens=2 delims=/" %a in ('aws s3 ls s3://bucketName/anyfolderif/ --recursive --region ap-northeast-1 ^| findstr ".aac$"') do echo %a,https://bucketName.s3.ap-northeast-1.amazonaws.com/anyfolderif/%a >> s3_urls.csv ***** You have to set your region this is my region (ap-northeast-1)
and if it not work then you have to install AWS CLI tool, https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html After installation do aws configure in your cmd(command prompt) it ask some key and secret key pin that you go through to any youtube video After configuration use the command that I give you in the first Remember that place you bucket_name or in you bucket have any folder then place the folder name to anyfolderif or one more change you region after that You can crate a csv file if you have a million of data or a short data nothing matter it create a csv file according to data

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions