- Newest
- Most votes
- Most comments
Q1. You can follow below steps to achieve your query of "query the S3 inventory configuration file for specific S3 objects and only send those specific objects to a Lambda which then puts the objects into a Datastream."
-
To create an inventory for a S3 bucket and store that inventory in a destination S3 bucket, please follow the steps mentioned in the document below https://linuxhint.com/create-inventory-configuration-s3-bucket/
-
In order to query the inventory file for specific S3 objects from destination S3 bucket using Athena, you may refer the following https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-inventory-athena-query.html
-
Once you have queried the desired objects store the resulting output file from Athena in a S3 bucket as described in below reference document https://docs.aws.amazon.com/athena/latest/ug/querying.html
-
After you have stored the results file of Athena to a S3 bucket, you can use that file to invoke your Lambda function, please refer to the below reference document https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html
-
To send the data to data stream from Lambda you can follow the documentation https://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html
Q2. To send S3 Batch Operations objects into a Data stream which will then forward them to Redshift
You may use S3 batch operations to invoke a Lambda function which will send it to the Data stream or Redshift based on your requirement. https://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-ops-invoke-lambda.html
Relevant content
- asked 5 months ago
- Accepted Answerasked 16 days ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 months ago
- AWS OFFICIALUpdated 6 months ago
- AWS OFFICIALUpdated a year ago