- Newest
- Most votes
- Most comments
You didn't miss anything here. As of today, to best of my knowledge to filter s3 objects on the basis of tags, it'd be a two step process:
- aws s3api list-objects-v2 --bucket <bucket_name> --profile <cli_profile> --query 'Contents[*].Key'
- Output content of above to a file
- Read though this file for each line and have following command run for each of those keys
- aws s3api get-object-tagging --bucket <bkt_name> --key <line_from_file> --profile <profile> --query 'TagSet[?Key==
<TagKey>] -> if this returns tag, store this key(s3object) to a file else skip that key(s3object)
Rresourcegroupstaggingapi will be used to find tags on resources but not on s3 objects as s3 files are data objects.
Hope you find this useful.
Note: Comment here if you don't get it working. I can give you couple of lines of code snippet to achieve this.
Abhishek
Hello,
After some investigations, please see below the solution written in Windows bash, in a .bat file. For some searched TagKey and TagValue, the composite command returns all the S3 Keys that have that Tag.
for /f "tokens=*" %%a in ('aws s3api list-objects-v2 --bucket <bucket> --prefix <prefix> --no-cli-auto-prompt --query Contents[*].[Key] --output text') do (
for /f "tokens=*" %%b in ('aws s3api get-object-tagging --bucket <bucket> --key %%a --no-cli-auto-prompt --query TagSet[?"Key=='<tagKey>'&&Value=='<tagValue>'"].Key --output text') do (
@echo %%a >> <file>
)
)
Thank you and have a good day,
Mihai ADAM
Hello,
Just to have a complete view on these comments, I've added a final version of the above architecture, that can also be found with comments on my site mihaiadam.com, application From Unstructured to Structured Data.
Have a good day,
Mihai
Relevant content
- asked 12 days ago
- asked 9 months ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated a year ago
Hello,
Thank you for your ideas and info about current status of AWS.
Because of the architecture of the application that I build at this moment (at home, for Proof Of Concept personal purposes), I already have in DynamoDB a list of the tags ready to be queried, built by some AI lambdas from some initial pictures, that in the end are tagged with those tags (please see below the diagram).
So, it is not appropriate to use the presented similar solution, and I will find a way to join the result from DynamoDB (s3_bucket_key,tag_key,tag_value) with S3 data (image objects links). The question was about the command to query only S3 without some extra references.
Thank you,
Mihai
Got it, there is no direct way. However, since you have list of tags stored in dynamodb, which you want to use to filter out s3 objects, I'd imagine why won't you be able to use a two step process in a lambda function per say to traverse your tag list from dynamodb and get s3 objects for those tags. However this would not be a cost effective solution.
Hello,
I was thinking about a way similar to that one described in this Question, using pipes from Windows cmd or Windows PS, but some strange error appears.
Thank you, Mihai
Hello,
One more thing please about this command : aws s3api get-object-tagging --bucket <bkt_name> --key <line_from_file> --profile <profile> --query 'TagSet[?Key==<TagKey>]
How could I also put a condition on the Value (and Key) of the TagSet (--query 'TagSet[?Key==<TagKey>]) ?
Thank you, Mihai
Sorry for delayed response. I'm glad you figured out for the value part, it'd be as same as tag key part. Do you have any additional questions, happy to assist.