By using AWS re:Post, you agree to the Terms of Use
AWS re:Postre:Post

How to move RDS Snapshot to S3 Glacier for long term storage

0

I have created an automated backup in RDS for a database instance to be backed up daily. But the max retention is only 35 days and I will like to keep the snapshots for 2 years for compliance reason. How do I move the snapshot from S3 into a S3 Glacier for long term storage?

Alternatively, I am thinking to use lambda to manually create snapshots so they won't have retention period apply on them. I will also like to move these snapshots to S3 Glacier. Again, how do I find out which S3 they are stored so I can apply lifecycle policy for them to be transferred to S3 Glacier after 6 months? This article stated that 'You can use snapshot-specific API calls to access the snapshots for copy and restore operations.' https://aws.amazon.com/blogs/database/amazon-rds-snapshot-restore-and-recovery-demystified/

Topics
Migration & ModernizationDatabase
Tags
Amazon Relational Database Service
Language
English
hannah
asked 18 days ago78 views
2 Answers
1

RDS snapshots are not stored in your buckets, they're stored in AWS-managed buckets (although you are still billed for the space they consume).

You can export the S3 data to one of your buckets https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ExportSnapshot.html

And then use the normal lifecycle management process to move to Glacier.

profile picture
EXPERT
Steve_M
answered 18 days ago
profile pictureAWS
EXPERT
Didier_Durand
reviewed 17 days ago
  • Didier_Durand EXPERT
    17 days ago

    Hi, yes, that's also how I would do to store archive in Glacier

  • hannah
    17 days ago

    The exported snapshots will be stored in the format of Parquet. Can I still restore the database instance with the exported snapshots? The database instances are MariaDB and Mysql. I tried exporting the snapshots of a test database of 20MB to S3, the process took a an hour. Our production DB is 30GB, so I am concerned with the time the export will take. Reading the Parquet format with Athena is not that straight forward: https://docs.aws.amazon.com/athena/latest/ug/parquet-serde.html. Looks like each DB table needs to be recreated before they can be queried

  • Steve_M EXPERT
    16 days ago

    The first half of this video here goes through all the steps https://www.youtube.com/watch?v=lyNGeDg6EII

    At 4:00 the panel at the top right appears to show that exporting 16GB took 47 minutes, this is with the database and target bucket both being in us-east-1.

    Is your target bucket in the same region as the database?

    Also take note of https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ExportSnapshot.html#USER_ExportSnapshot.Exporting

    The time it takes for the export to complete depends on the data stored in the database. For example, tables with well-distributed numeric primary key or index columns export the fastest. Tables that don't contain a column suitable for partitioning and tables with only one index on a string-based column take longer. This longer export time occurs because the export uses a slower single-threaded process.

0

Hello.

Automatic backups of RDS are usually sent to S3 managed by AWS.
Therefore, please follow the steps in the document below to export automatic snapshots to S3 that you manage.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ExportSnapshot.html#USER_ExportSnapshot.Overview

profile picture
EXPERT
Riku_Kobayashi
answered 18 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content