Skip to content

Need recommendations around RDS back up in production account and restore to non-production account

0

Hello Experts, I am moving my critical business application to AWS and struggling with setting up the efficient back and restore procedure. It's a SQL server with 4 databases.

Current On-prem process.

  • Take full weekly back up on Sunday
  • Take differential backup every business day before and after the nightly batch cycle
  • We also use Delphix which can provide us the flexibility to restore the database in 15 mins interval. Delphix team would have their own way to taking back up but they are able to provide us the restore point in 15 mins interval. They must be taking full + differential or transaction log combo

since, we are moving to AWS, we want same kind of flexibility. We have 4 aws accounts production, stage, qa and dev. Production will have one RDS instance which will have 4 databases on it. Its multi AZ enabled.

We want the following solutions... 1)Our production support team wants the data which is provided by Delphix above (maximum of 15 mins lag to production). This is needed for troubleshooting. Our db sizes are 1.2TB, 500GB, 100GB and 50 GB. This needs to be in the non-production environment 2)Dev and QA should be ok with pre and post cycle back up. This needs to be in the non-production environment. Essentially, if above requirement can be fulfilled then this is done as well.

Thanks in advance. Avinash.

asked a year ago529 views

3 Answers
0

Based on your requirements for moving from on-premises to AWS RDS for SQL Server, here are recommendations for your backup and restore procedures across accounts:

For your production environment:

  1. Enable automated backups on your RDS SQL Server instance. Amazon RDS performs daily backups during your defined backup window and uploads transaction logs to S3 every 5 minutes. You can configure retention for up to 35 days.

  2. For your 15-minute restore point requirement (similar to Delphix functionality), RDS's point-in-time recovery (PITR) capability will be valuable. Since RDS uploads transaction logs every 5 minutes, you can restore to any point in time within your retention period, up to the latest restorable time (typically the last five minutes).

  3. Take manual snapshots before and after your nightly batch cycles. These snapshots persist until you explicitly delete them and provide an additional layer of protection for critical processing periods.

For cross-account restoration to non-production environments:

  1. Set up a cross-account backup solution using AWS Backup. This involves:
  • Creating AWS KMS customer managed keys and backup vaults in both production and non-production accounts
  • Creating a backup plan in your production account to schedule the RDS database backups
  • Configuring the backup plan to copy backups to your non-production accounts (stage, QA, dev)
  1. For SQL Server specifically, you can also leverage native backup and restore functionality:
  • Enable the SQLSERVER_BACKUP_RESTORE option on your RDS option group
  • Create full or differential backups of your databases and store them in S3
  • Restore these backups to your non-production RDS instances

This approach will allow your production support team to access data with minimal lag (meeting your 15-minute requirement) in non-production environments for troubleshooting, while also providing the pre and post cycle backups needed for dev and QA environments.
Sources
Backup and recovery for Amazon RDS - AWS Prescriptive Guidance
Automate cross-account backup of Amazon RDS for Oracle including database parameter groups, option groups and security groups | AWS Database Blog
Support for native backup and restore in SQL Server - Amazon Relational Database Service
Backing up and restoring your Amazon RDS DB instance - Amazon Relational Database Service
Community | Architecting for Zero Data Loss Disaster Recovery using Amazon RDS Solutions

answered a year ago

  • We would prefer native back up solution because this way, we can take db level back up rather than instance level but with 5 multi file options, it took 9 hours for my 1.2 TB database. This is not acceptable because it reduces my batch cycle window. Is there a solution to minimize the back up time.

0

Migrating your critical SQL Server application to AWS RDS requires a shift in thinking from traditional on-prem tools like Delphix, but you can absolutely achieve and even exceed your required restore flexibility using native AWS features. The key is to combine two core RDS capabilities: Point-in-Time Restore (PITR) for your production support team's high-frequency needs, and Manual Snapshots for your Dev/QA batch cycle requirements.

To address your first and most critical requirement providing the support team with a database copy with a maximum 15-minute lag, you will rely on RDS's automated backup and Point-in-Time Restore functionality. When you enable automated backups on your Multi-AZ production instance, RDS continuously archives your transaction logs to S3. This gives you a Recovery Point Objective (RPO) of approximately 5 minutes, easily meeting your 15-minute target. For troubleshooting, your support team can initiate a PITR to any specific second within your backup retention period (up to 35 days). It is crucial to understand that while your restore point is available within minutes, the restore time (RTO) for large databases like your 1.2TB instance will be longer than 15 minutes, as RDS must create and hydrate an entirely new instance. The process would be to restore a PITR copy of the production database directly into your non-production AWS account as a new, temporary RDS instance whenever troubleshooting is needed.

For your second requirement providing Dev and QA with pre- and post-batch cycle copies the ideal solution is using RDS Manual Snapshots. You can easily automate this process by using an AWS Lambda function triggered by Amazon EventBridge to create a named manual snapshot of your production instance immediately before your nightly batch cycle begins and another one immediately after it finishes. These snapshots are stable, point-in-time images of your database. You can then share these manual snapshots with your other AWS accounts (dev, qa, stage). Your development and QA teams can then restore a new RDS instance from the appropriate pre- or post-batch snapshot whenever they need it, giving them a perfect, isolated environment for their work. This automated snapshot strategy fully covers the needs of your development lifecycle and is a clean, cost-effective way to manage predictable restore points.

answered a year ago

  • Hello Arvind, Appreciate the time you took to provide recommendation. Wanted to ask you a follow up question on the first requirement "providing the support team with a database copy with a maximum 15-minute lag". You said "The process would be to restore a PITR copy of the production database directly into your non-production AWS account as a new, temporary RDS instance whenever troubleshooting is needed." at the end of your response. Is this possible? As far as I read, its not possible to do a PITR from aws automated back up from production account to the non production account?

    Absolutely agree and aligned with the second requirement and we started implementing that already.

    I would appreciate if you would respond to the follow up.

0

Thanks Avinash, for clarifying your requirement.

I understand that you want to take DB level backup and not instance level. While you tried DB level backup, timing of backup is your main concern. I would like to suggest few parameters to improve backup. Kindly retry one more time with below parameters and see if this help to optimize the backup timing.

=> Key parameter optimization:

- Use maximum @number_of_files=10 for parallel processing
- Set @max_transfer_size to 4194304 (4MB) for larger chunks
- Use larger @block_size=65536 for better I/O performance
- Increase @buffer_count for more I/O buffers

You can refer below document for more details on same and also go through limitations, to be more aware about all use cases and parameter limitations.

[+] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.Native.Using.html#SQLServer.Procedural.Importing.Native.Using.Backup

Hope it helps and if it does, I would appreciate if answer can be accepted so that community can benefit for clarity when searching for similar enquiries in repost/aws guide.

AWS
SUPPORT ENGINEER

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.