RDS: Unexpected Postgres Upgrade & Downgrade Plan

Question

I am writing to address a critical issue regarding our Production RDS database instance ([db.m5.xlarge, Postgres]).

Unexpected Upgrade:

We were recently shocked to discover that Postgres has automatically upgraded to version 14.7 without prior notice. This is concerning because, as per our latest successful Terraform apply (v0.13), all environments should be on version 10.14.

Investigation:

I have the following questions regarding this unexpected upgrade:

* Root Cause: What could have triggered this major version upgrade? Could an automatic upgrade flag have been inadvertently set? As per the documentation (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_UpgradeDBInstance.PostgreSQL.html), I understood major upgrades require manual intervention.
* Audit Trail: Is there any way to track the upgrade in the AWS console or logs, including the exact Zeitpunkt and process? Understanding the upgrade path would be immensely helpful.
* Downgrade Plan: We are currently planning a staged upgrade for all RDS instances to Postgres 13.x. Consequently, we need to downgrade our Production instance from 14.7 to 13.x. Given that db.m5.xlarge supports versions 11, 13, and 15, could we simply modify the Postgres version in Terraform tf file and apply the changes? If so, could you please share any potential pitfalls or recommended AWS documentation for a smooth downgrade process?

Answer

Hello.

> Root Cause: What could have triggered this major version upgrade? Could an automatic upgrade flag have been inadvertently set?

If you look at the document below, support for PostgreSQL 10.14 has already ended.  
AWS may automatically upgrade versions that are no longer supported.  
https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-release-calendar.html

The following documents are related to PostgreSQL10.  
https://repost.aws/questions/QUph1IFLkkRiyc0pCdTH493Q/announcement-amazon-rds-for-postgresql-10-deprecation  
> Starting February 14, 2023 00:00:01 AM UTC, you will not be able to create new RDS instances with PostgreSQL major version 10 from either the AWS Console or the CLI. We recommend you to upgrade your databases before April 17, 2023. RDS will upgrade your PostgreSQL 10 databases to PostgreSQL 14 during a scheduled maintenance window between April 17, 2023 00:00:01 UTC and July 18, 2023 00:00:01 UTC. On July 18, 2023 00:00:01 AM UTC, any PostgreSQL 10 databases that remain will be force upgraded to version 14 regardless of instances’ scheduled maintenance window.

> Audit Trail: Is there any way to track the upgrade in the AWS console or logs, including the exact Zeitpunkt and process?

Basically, AWS will notify you if you are using a deprecated version.  
You can probably confirm that the upgrade has taken place by looking at the logs.  
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html

> Downgrade Plan: We are currently planning a staged upgrade for all RDS instances to Postgres 13.x. Consequently, we need to downgrade our Production instance from 14.7 to 13.x. Given that db.m5.xlarge supports versions 11, 13, and 15, could we simply modify the Postgres version in Terraform tf file and apply the changes? If so, could you please share any potential pitfalls or recommended AWS documentation for a smooth downgrade process?

I don't think there is an official way to downgrade on the AWS side.  
Therefore, you will need to use a database migration service such as DMS to migrate to a lower version of RDS PostgreSQL.  
https://docs.aws.amazon.com/dms/latest/userguide/Welcome.html

RDS: Unexpected Postgres Upgrade & Downgrade Plan

Relevant content