- Newest
- Most votes
- Most comments
Create a CMD file with aws s3 sync CLI commands to copy the files to Amazon S3 bucket like example below. Create a scheduled task in Windows to periodically execute the CMD file.
aws s3 sync C:\folder. s3://bucket-name/
If you use aws s3 sync, then as the Windows folder is updated, the sync will automatically update S3.
Adding to the other answer, two things I would also do:
Create a role for the instance that allows it to access S3. That way you won't have to create long-term credentials that go on the instance - they are automatically learned by the CLI tool from the instance metadata service. You can also restrict which buckets the instance can access if you need to.
Second, create a S3 Gateway Endpoint in your VPC. It is free of charge and depending on your network setup and the amount of logs you transfer it can save you data transfer fees.
Hi Brettski-AWS,
Thanks for the update, Could you please provide the some example link to sync data from server to S3 by VPC endpoint. I have seen this link also shared by you but could you please provide the some example for this ?
Thanks,
You don't need to configure anything to use the endpoint - that's the beauty of it. Once it is configured for the VPC and the route tables updated (you do this when you create the endpoint) you're good to go.
Relevant content
- Accepted Answerasked 9 months ago
- asked 2 years ago
- asked 2 years ago
- asked 2 months ago
AWS OFFICIALUpdated 8 months ago- AWS OFFICIALUpdated 3 months ago
- AWS OFFICIALUpdated 6 months ago
- AWS OFFICIALUpdated a year ago
Thanks Brian_Klein for the update.
Is there any solution for automatic sync the s3 bucket when new files come into the local server. I do not want to set scheduler that run in some interval.
The aws s3 sync command updates S3 when local folder is updated and once you run the command, it continues to run as a process. You need a mechanism to execute the command and monitor it which is where running as a scheduled task is useful.