RDS MySQL stuck offline & upgrading

0

One of our DB instances on MySQL 8.0.29 was crashing with regular "DB instance restarted" in the logs. I tried modifying it to 8.0.30, but now it's stuck upgrading, with regular "Error restarting mysql: MySQL process is not running and the restart attempt failed." in the logs.

The error log contains the following over and over again:

2022-09-23T16:03:46.082336Z 0 [Warning] [MY-011068] [Server] The syntax 'log_slave_updates' is deprecated and will be removed in a future release. Please use log_replica_updates instead.
2022-09-23T16:03:46.082352Z 0 [Warning] [MY-011069] [Server] The syntax '--master-info-repository' is deprecated and will be removed in a future release.
2022-09-23T16:03:46.082362Z 0 [Warning] [MY-011069] [Server] The syntax '--master-info-repository' is deprecated and will be removed in a future release.
2022-09-23T16:03:46.082415Z 0 [Warning] [MY-011069] [Server] The syntax '--relay-log-info-file' is deprecated and will be removed in a future release.
2022-09-23T16:03:46.082425Z 0 [Warning] [MY-011069] [Server] The syntax '--relay-log-info-repository' is deprecated and will be removed in a future release.
2022-09-23T16:03:46.082457Z 0 [Warning] [MY-011068] [Server] The syntax 'skip_slave_start' is deprecated and will be removed in a future release. Please use skip_replica_start instead.
2022-09-23T16:03:46.082469Z 0 [Warning] [MY-011068] [Server] The syntax 'slave_exec_mode' is deprecated and will be removed in a future release. Please use replica_exec_mode instead.
2022-09-23T16:03:46.082483Z 0 [Warning] [MY-011068] [Server] The syntax 'slave_load_tmpdir' is deprecated and will be removed in a future release. Please use replica_load_tmpdir instead.
2022-09-23T16:03:46.082676Z 0 [Warning] [MY-010101] [Server] Insecure configuration for --secure-file-priv: Location is accessible to all OS users. Consider choosing a different directory.
2022-09-23T16:03:46.082703Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead.
2022-09-23T16:03:46.082719Z 0 [System] [MY-010116] [Server] /rdsdbbin/mysql/bin/mysqld (mysqld 8.0.29) starting as process 726
2022-09-23T16:03:46.085105Z 0 [Warning] [MY-010161] [Server] You need to use --log-bin to make --log-replica-updates work.
2022-09-23T16:03:46.089322Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2022-09-23T16:03:46.095259Z 1 [Warning] [MY-012191] [InnoDB] Scan path '/rdsdbdata/db/innodb' is ignored because it is a sub-directory of '/rdsdbdata/db/
2022-09-23T16:03:48.212117Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.

InnoDB: Progress in percents: 116:03:49 UTC - mysqld got signal 11 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
Thread pointer: 0x1518a5406000
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 15189fcc63e8 thread_stack 0x40000
/rdsdbbin/mysql/bin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x2e) [0x2097dae]
/rdsdbbin/mysql/bin/mysqld(print_fatal_signal(int)+0x2bb) [0x102057b]
/rdsdbbin/mysql/bin/mysqld(handle_fatal_signal+0xa5) [0x1020625]
/lib64/libpthread.so.0(+0x118e0) [0x1518c0d458e0]
/rdsdbbin/mysql/bin/mysqld(page_cur_search_with_match_bytes(buf_block_t const*, dict_index_t const*, dtuple_t const*, page_cur_mode_t, unsigned long*, unsigned long*, unsigned long*, unsigned long*, page_cur_t*)+0x190) [0x22d69b0]
/rdsdbbin/mysql/bin/mysqld(btr_cur_search_to_nth_level(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_cur_t*, unsigned long, char const*, unsigned long, mtr_t*)+0x1ec2) [0x2406642]
/rdsdbbin/mysql/bin/mysqld(row_search_on_row_ref(btr_pcur_t*, unsigned long, dict_table_t*, dtuple_t const*, mtr_t*)+0xd2) [0x233d2c2]
/rdsdbbin/mysql/bin/mysqld(row_undo_search_clust_to_pcur(undo_node_t*)+0x15f) [0x234bd4f]
/rdsdbbin/mysql/bin/mysqld(row_undo_mod(undo_node_t*, que_thr_t*)+0x18f) [0x258461f]
/rdsdbbin/mysql/bin/mysqld(row_undo_step(que_thr_t*)+0x52) [0x234c542]
/rdsdbbin/mysql/bin/mysqld(que_run_threads(que_thr_t*)+0x5c8) [0x22f3da8]
/rdsdbbin/mysql/bin/mysqld(trx_rollback_or_clean_recovered(bool)+0x9a5) [0x23a4475]
/rdsdbbin/mysql/bin/mysqld(trx_recovery_rollback_thread()+0x1c) [0x23a53fc]
/rdsdbbin/mysql/bin/mysqld(std::thread::_State_impl<std::thread::_Invoker<std::tuple<Detached_thread, void (*)()> > >::_M_run()+0xb0) [0x2296fc0]
/rdsdbbin/mysql/bin/mysqld() [0x2baabf0]
/lib64/libpthread.so.0(+0x744b) [0x1518c0d3b44b]
/lib64/libc.so.6(clone+0x3f) [0x1518c052040f]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0): Connection ID (thread ID): 0
Status: NOT_KILLED

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

Is there any way to fix this, or are we going to have to delete / recreate the instance?

gefragt vor 2 Jahren117 Aufrufe
Keine Antworten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen