How to move RDS Snapshot to S3 Glacier for long term storage
I have created an automated backup in RDS for a database instance to be backed up daily. But the max retention is only 35 days and I will like to keep the snapshots for 2 years for compliance reason. How do I move the snapshot from S3 into a S3 Glacier for long term storage?
Alternatively, I am thinking to use lambda to manually create snapshots so they won't have retention period apply on them. I will also like to move these snapshots to S3 Glacier. Again, how do I find out which S3 they are stored so I can apply lifecycle policy for them to be transferred to S3 Glacier after 6 months? This article stated that 'You can use snapshot-specific API calls to access the snapshots for copy and restore operations.' https://aws.amazon.com/blogs/database/amazon-rds-snapshot-restore-and-recovery-demystified/
- Newest
- Most votes
- Most comments
RDS snapshots are not stored in your buckets, they're stored in AWS-managed buckets (although you are still billed for the space they consume).
You can export the S3 data to one of your buckets https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ExportSnapshot.html
And then use the normal lifecycle management process to move to Glacier.
Hello.
Automatic backups of RDS are usually sent to S3 managed by AWS.
Therefore, please follow the steps in the document below to export automatic snapshots to S3 that you manage.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ExportSnapshot.html#USER_ExportSnapshot.Overview
Relevant content
- Accepted Answerasked 2 years ago
- asked 5 months ago
- Accepted Answerasked 3 months ago
AWS OFFICIALUpdated 2 years ago- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
Hi, yes, that's also how I would do to store archive in Glacier
The exported snapshots will be stored in the format of Parquet. Can I still restore the database instance with the exported snapshots? The database instances are MariaDB and Mysql. I tried exporting the snapshots of a test database of 20MB to S3, the process took a an hour. Our production DB is 30GB, so I am concerned with the time the export will take. Reading the Parquet format with Athena is not that straight forward: https://docs.aws.amazon.com/athena/latest/ug/parquet-serde.html. Looks like each DB table needs to be recreated before they can be queried
The first half of this video here goes through all the steps https://www.youtube.com/watch?v=lyNGeDg6EII
At 4:00 the panel at the top right appears to show that exporting 16GB took 47 minutes, this is with the database and target bucket both being in us-east-1.
Is your target bucket in the same region as the database?
Also take note of https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ExportSnapshot.html#USER_ExportSnapshot.Exporting