By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Aurora PostgreSQL Upgrade (12.16 to 15.4) via Blue/Green not finishing

0

Trying a dry run for version upgrade withe bg deployments (see background info below). This is stuck for many hours.

Question:

  • How can i find more information about why its stuck
  • Any tips are also welcome

Cheers!

  • The green cluster has been on "Upgrading" state for ~20hours.
  • There are no logs/events written since "Upgrade in progress: Upgrading writer."
  • CPU has flat lined at 2% for Past 15hrs
  • Freeable memory has flat lined at 300GB for the past 15hrs
  • NetworkThruput was only around 40mb/s (the instance has a max thruput of >2000m/s ) at that rate it will take 40hrs to pg_dump the whole database. Can this be improved? at this rate it will take 40hrs to pg_dump all the data.
  • Database stats :
    • has >900 Tables with ~6TB in size
    • db.r6g.16xlarge single instance cluster
Topics
Migration & ModernizationAnalyticsDatabase
Tags
Amazon Relational Database ServiceAurora PostgreSQLDatabaseAmazon Aurora
Language
English
Ruben
asked 3 months ago270 views
1 Answer
0

Hi,

Follow this guidance for troubleshooting such a major version upgrade: https://repost.aws/knowledge-center/rds-postgresql-version-upgrade-issues

Did you try another strategy with AWS DMS ?

  1. construct a fresh instance of RDS with v15.4
  2. use AWS DMS bulk migration features to migrate the data from old to new

If you do that, you will have another advantage for the real migration: you can stay on old version and replicate live via the CDC features of DMS over some period of time to see if the new version doesn't have issues when new data is added to the db

Best,

Didier

profile pictureAWS
EXPERT
Didier_Durand
answered 3 months ago
  • Ruben
    3 months ago

    Hi @Didier Thanks for the reply, I tried these suggestion in the link above but this part is already take care when you create a B/G deployment. Also this environment is completely detached from our application and therefore has has nothing running in it. DMS was our initial approach but this didn't work out for this particular database because of its high transactions/sec.

    Note: An interesting thing was that the NetworkThruput was only around 40mb/s (the instance has a max thruput of >2000m/s ) at that rate it will take 40hrs to pg_dump the whole database. Can this be improved?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content