By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Mitigating downtime for blue-green deployments / recommended path for encrypting dbs

0

Hi, we have time-sensitive upgrades for our rds instances that are on versions that will be put on EOL soon.

1a) We need to mitigate downtime during upgrades. Is there a way to disable the write pausing that occurs prior to the dns update?

1b) Otherwise - we would prefer to point to the green database new URL endpoint and remove the blue database instead of executing the blue-green switchover to avoid downtime. Is there any reason we cannot do that, or advice on how to best do that?

  1. From what I understand, encryption of unencrypted database instances is not supported for blue-green deployments. Our best workaround idea for no-downtime migration right now is to create a new empty encrypted instance ourselves and enable logical replication to transfer the data, then cut over to it. Would that work? Are there better supported paths? I understand we can generate a snapshot and copy to an encrypted snapshot but we would need to do a more complicated logical replication operation to start replication from the correct LSN.
Topics
Migration & ModernizationDatabaseAnalytics
Tags
AWS Database Migration ServicePostgreSQLAmazon Relational Database ServiceEncryption
Language
English
echngmx
asked a month ago50 views
1 Answer
0

Hello. Thats a good questions about RDS. Lets deep dive in to them 1. a) Unfortunately you cannot disable write pausing during blue-green switchover because this is build-in option in AWS. b) Of course you can do manual cutover instead of triggering the blue-green switchover. So basically you can create your own green database manually or with Blue/Green deployment tooling without switching and then point to your new database endpoint manually (update your app config or secrets/SSM params

  1. Yes, thats correct. RDS Blue/Green does not support encrypting an unencrypted instance as part of the upgrade, and you can’t change the encryption state. So I think your workaround is good and you can do it:
  • Create a new encrypted RDS instance with the desired engine version.
  • Set up logical replication (for PostgreSQL or MySQL) from the unencrypted to the encrypted instance.
  • Once caught up, perform the cutover manually by:
  • Stopping writes to the source DB.
  • Ensuring the target is in sync.
  • Switching traffic to the new DB.
profile picture
NetworkPitu
answered a month ago
  • echngmx
    a month ago

    b) Of course you can do manual cutover instead of triggering the blue-green switchover. So basically you can create your own green database manually or with Blue/Green deployment tooling without switching and then point to your new database endpoint manually (update your app config or secrets/SSM params

    When I look into this further, the green instances are considered read-only until switchover is complete. There is some wiggle room to enable writes manually if the instances are using logical replication but in our case the default mode of replication would be physical (v16 minor version upgrade). There also seems to be a "Promote" option, I assume this enables writes on the instance but it also breaks replication so there's no way to shift write traffic to the green replica before replication is halted. Finally, I don't see any option to specify the desired db identifier for the green replica either during the blue-green deployment or when promoting the replica (please correct me if wrong) so I think we'd be stuck with {current-db-identifier}-green-{randomstring} permanently which is a little unfortunate (there is an option to rename the db instance but that also causes downtime).

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content