RDS instance keeps rebooting

0

I have a MySQL RDS instance db.m4.xlarge, Multi-AZ, 10379 GiB Storage. 1100 GiB free space.
In this database I have 3 tables that are over 1 TB each and 30 or so tables of 1GB or much less.
It started throwing errors like this:

2019-02-02 18:31:53 2b740ec02700 InnoDB: Error: space id and page n:o stored in the page
InnoDB: read in are 10621440:285212834, should be 552:17605632!
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 17605632.
InnoDB: You may have to recover from a backup.

And rebooting the instance every few minutes.
I created a read replica, and promoted that replica, then moved all the traffic to that new instance, and removed the old one. That seemed to work for about a day, but then the errors returned.
I then created a snapshot, and restored that snapshot into a new instance, which worked for about a day, and then the errors returned.
I am accessing the database through JDBC. Using standard SQL queries, no triggers or stored procs.

I could use some suggestions.
Thanks!

Edited by: dustin8192 on Feb 2, 2019 11:07 AM

demandé il y a 5 ans378 vues
2 réponses
0

It seems likely that you have a data inconsistency in the innodb tables space. This gets propagated into a new read replica or a restore from backups. I'd recommend you use mysqldump to extract data from your instance and reload it back into a new instance to fix the metadata inconsistency.

-Phil

AWS
MODÉRATEUR
philaws
répondu il y a 5 ans
0

There was a corrupted index on a table. reading or writing a specific row in the table would cause the server to reboot. I created a new database from scratch from the code, and then transferred all records semi-manually using MANY mysqldump statements over several weeks - the db is 11TB.
No problems since.

répondu il y a 4 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions