RDS MySQL 8.4 Upgrade - Current Parameter Group is non-default

When upgrading from MySQL 8.0 to 8.4 in Amazon RDS, you're encountering an expected behavior regarding parameter groups. The error message is indicating that you need to explicitly specify a parameter group that's compatible with MySQL 8.4 during the upgrade process.

This isn't a contradiction with your utf8mb4 character set configuration - in fact, you're correct that using utf8mb4 is recommended for MySQL 8.4 upgrades since utf8mb3 is deprecated. The issue is that parameter groups are version-specific in RDS, and you need to create a new parameter group for the 8.4 engine version.

Here's how to resolve this:

1. Create a new parameter group that's compatible with MySQL 8.4

• Navigate to the RDS console parameter groups section
• Click "Create parameter group"
• Select MySQL as the engine type
• For Parameter group family, select the version that matches your upgraded engine (mysql8.4)
• Provide a name and description
• Click "Create"

2. Configure the new parameter group with your custom settings

• Copy your current custom parameters to the new parameter group:
• character_set_client: utf8mb4
• character_set_server: utf8mb4
• collation_connection: utf8mb4_0900_ai_ci
• collation_server: utf8mb4_0900_ai_ci
• max_connections: 1000
• time_zone: Australia/Brisbane

3. When you attempt the upgrade again, explicitly specify this new parameter group in the upgrade request.

4. After the upgrade completes, you may need to reboot the instance to apply all parameter group changes. The instance's parameter group status will show 'pending-reboot' if this is necessary.

This approach allows you to maintain your required settings (including utf8mb4 character sets and your time zone configuration) while satisfying the requirement to use a parameter group that's compatible with the new engine version.
Sources
RDS Engine Upgrade Failure – Parameter Group Family Mismatch | AWS re:Post
Major version upgrades for RDS for MariaDB - Amazon Relational Database Service
Testing an RDS for MySQL upgrade - Amazon Relational Database Service
Major version upgrades for RDS for MySQL - Amazon Relational Database Service