Aurora RDS MySQL 5.6.10a (1.14.1) Upgrade to 5.7 (2.x) and eventually 8.0 (maybe)

Question

Hello,

I have an 8TB Aurora MySQL database on RDS with 2 read replicas. Current db engine version is 5.6.10a (1.14.1).

Instances:
db.r3.2xlarge
db.r3.2xlarge
db.r3.xlarge

I was looking to upgrade this to at least 1.22.3 so that I could do a green-blue upgrade process to get it to 5.7 (2.x) based on this: https://aws.amazon.com/blogs/database/performing-major-version-upgrades-for-amazon-aurora-mysql-with-minimum-downtime/

I was wondering if the upgrades from 1.14.1 to 1.22.3 are "disk intensive", or are they purely "software updates" so to speak.

I can't find any documentation to indicate anything either way, and I'm hesitant to initiate something I cannot cancel, and have our production environment down for hours/days in the process.

Would I be better off creating a clone and upgrading the clone to get a feel for the times? (Ignoring the fact that it won't have any load on it and may not be an exact representation of speed)

Thanks so much!

Accepted Answer

Hi mshussein, to complement the tips YoungCTO_quibski provided above:

Differently from the [major version](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.VersionPolicy.html#Aurora.VersionPolicy.MajorVersions) upgrade (1.x to 2.x, and 2.x to 3.x), the upgrade from 1.14.1 to 1.22.3 is not "disk intensive" as it is a minor upgrade. By the way, the [latest patch currently available for the minor version 1.22 is 1.22.5](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.1225.html), so I'd recommend going to 1.22.5 instead of 1.22.3 due to the additional bugs fixed.

But, although the minor upgrade is not disk intensive, it includes downtime, and anytime a MySQL database is stopped and started, the database will execute startup procedures (also during crash recovery), that take longer the more transactions, active sessions, and read views were open in the database. I'd recommend that before initiating any upgrade procedure, you connect to the database and
- Check the history list length (the HLL should not be larger than a few thousand) using [the procedure described here](https://aws.amazon.com/premiumsupport/knowledge-center/aurora-mysql-slow-select-query/).
- Check that there are no long-running active sessions (especially write sessions like DCLs, DDLs and DMLs) by running `SHOW PROCESSLIST;` or having a look at Performance Insights, if it is active for your instance.

In a cluster with no active transactions, no active sessions, and a catched-up purge, the whole minor upgrade procedure normally takes several minutes, but its downtime normally lasts less than a minute.

However, I'd still first test the whole procedure in a cluster restored from a snapshot or cloned, to be sure - this is also mentioned [in the documentation](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.VersionPolicy.html#Aurora.VersionPolicy.PreupgradeTesting).

Answer

When upgrading major versions, there is an expected downtime. And once started it cannot be cancelled anymore.  What you can do is do a gameday/simulation: Restore a snapshot to another RDS then use that as test for your upgrade.

Another option for limited downtime is using DMS replication as well, but a new endpoint is generated. Hopefully your endpoints are not hardcoded.

Aurora RDS MySQL 5.6.10a (1.14.1) Upgrade to 5.7 (2.x) and eventually 8.0 (maybe)

Relevant content