How can I Transfer old data from FSX for Windows to S3 using Datasync filtering by date

Question

I'm trying to move cold data from FSX for Windows to S3 standard storage and eventually move to S3 Glacier Deep Archive using the lifecycle rules in S3. I am trying to filter data by date, basically moving everything prior to 01-01-2023 to S3.  I've tried using - /**.* [LastModifiedTime>=2023-01-01] in the excludes filtering but it doesn't seem to be applying the filter.  Is there a different string I should be trying in the exclude filtering or is there another way I should be handling this.

Giovanni Lauria · Answer

AWS DataSync can be used to move data from FSX for Windows to S3 and eventually to S3 Glacier Deep Archive. However, DataSync doesn’t support filtering by last modified date. To filter data by date, you might need to create a custom script or application that lists all the files in your FSX file system, filters them based on the last modified date, and then uses DataSync or the AWS SDK to transfer the files to S3. Once the data is in S3, you can use S3 Lifecycle policies to transition the data to S3 Glacier Deep Archive.

How can I Transfer old data from FSX for Windows to S3 using Datasync filtering by date

Add your answer

Relevant content