ok... I did find out how to deal with it... First, you really need to upgrade it in baby steps. This is my writeup

PosgreSQL + PostGIS, from version 9.5 to 13.3

First you need to go to the latest 9.5.25, this can be easily done using a simple AWS CLI modify command in your DB instance. Now update POSTGIS and any other extensions you have to the newest one, if in doubt, look up the newest version.

I made a python script for myself that does update all DBs I have with the extension version I ask.

postgis_version='2.2.5' postgis_topology_version='2.2.5'

Next step, do change your version to 9.6.22, also change the machine the DB is running to something newer, this will help in the next steps (newer DB versions cannot run in older instances).

postgis_version='2.5.2' postgis_topology_version='2.3.7'

Now you may upgrade to version 10.17.

postgis_topology_version='2.5.2'

Once upgrade to version 10.17 is done, you may upgrade to 11.12... I think you can upgrade to 11.12 directly, without the 10.17 stop, but I was in no rush (and it was quick anyway).

At version 11.12, you may now upgrade POSTGIS and any others.

postgis_version='2.5.2next' postgis_topology_version='2.5.2next'

Now you may upgrade to version 11.13, this will enable you to install version POSTGIS at version 3.1.4, allowing you to go up to postgres 13.3

postgis_version='3.1.4' postgis_topology_version='3.1.4'

And you are done. I hope this helps you.

Also, if you have multiple databases in your instance, make sure all of them have the latest extensions before upgrading, and always run SELECT * from pg_available_extension_versions where name='postgis'; to find which are the latest versions of the extensions before updating.