Amazon Macie - Daily Job run to pick only new/modified files added in last 24 hrs

How about this:

1. Create a Scheduled Discovery Job o Go to the Macie console → Discovery jobs → Create job o Choose the S3 buckets you want to scan
2. Set the Job Frequency o Choose Daily under “Schedule” o This ensures the job runs every 24 hours
3. Refine Scope with Object Criteria o Under Additional settings, enable Object criteria filters o Set the filter:  Last modified date → is after → NOW - 1 day  This tells Macie to only scan files modified in the last 24 hours
4. Save and Activate the Job o Review your settings and launch the job o Macie will now scan only the files that were added or changed since the last run

https://docs.aws.amazon.com/macie/latest/user/discovery-jobs.html

Hi Anshuman. Thanks for reaching out with your question to AWS re:Post!

To configure a Macie sensitive data discovery job run to pick up only new or modified files added since the previous run on a daily basis:

When creating a sensitive data discovery job in the AWS Management Console, you can configure these settings on Step 3: Define the schedule and refine the scope.

To run the job periodically on a recurring basis, choose Scheduled job. For Update frequency, choose to run the job daily for your use case. Then select the Include existing objects checkbox to analyze all existing objects immediately after you finish creating the job. Each subsequent run analyzes only those objects that are created or changed after the preceding run. Macie will automatically handle tracking when the last run was performed and scanning for new or modified files to scan.

For more information about Step 3: Define the schedule and refine the scope, see: https://docs.aws.amazon.com/macie/latest/user/discovery-jobs-create.html#discovery-jobs-create-step3.