AWS Datasync to transfer the PDF and xml togather

Question

Dear Team,

We have a source system on Premise DC which will upload thousands of PDF and its metadata XML files in Windows NAS.   We need to upload the same to S3 and further process it.  To process, we need to make sure that the pdf and metadata xml file are both available on S3.

I wanted to check if there is a way in AWS Datasync which will make sure that PDF and its corresponding Metadata xml file(both will have the same name but different extensions) both are loaded to S3 togather so that we can start the processing as soon as the files are uploaded to S3.

Thank you in advance.

Regards,
Dhaval Mehta

Answer

By default, DataSync will copy all files in a directory.  So in your case, both PDF and XML files will be transferred.  If you want to trigger your downstream processing only after all files have been copied, then you can monitor for SUCCESS status on the Task Execution CloudWatch event.  For more information, see https://docs.aws.amazon.com/datasync/latest/userguide/monitor-datasync.html#events.

AWS Datasync to transfer the PDF and xml togather

Relevant content