AWS Lightsail - instance down after maintenance

0

I am using AWS lightsail in the US West 2 region. Maintenance us scheduled today 21st June at GMT - 4:00 for 2 hours. Now my instance is down and it has bee over 12 hours. I was taking snapshot of the instance, and even the snapshots are not working. I am able to SSH into the instance but somehow the public IP does not show anything. Please help me resolve this issue The instance is Bitnami wordpress.

  • Apparently, on some refresh the error changes to Error establishing a database connection. Please guide me how to resolve this. I do see one SQL process running by the name of bitnami when i do ps aux | grep mysql. There is also enough disk space. And i tried rebooting my instance multiple times

  • some error logs: 2023-06-21 18:17:18 0 [ERROR] InnoDB: Page [page id: space=0, page number=50] log sequence number 4200479366 is in the future! Current system log sequence number 3542625345. 2023-06-21 18:17:18 0 [ERROR] InnoDB: Your database may be corrupt or you may have copied the InnoDB tablespace but not the InnoDB log files. Please refer to https://mariadb.com/kb /en/library/innodb-recovery-modes/ for information about forcing recovery. 2023-06-21 18:17:18 0 [Note] InnoDB: Retry with innodb_force_recovery=5 2023-06-21 18:17:18 0 [ERROR] InnoDB: Plugin initialization aborted with error Data struct ure corruption 2023-06-21 18:17:18 0 [Note] InnoDB: Starting shutdown... 2023-06-21 18:17:18 0 [ERROR] Plugin 'InnoDB' init function returned error. 2023-06-21 18:17:18 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2023-06-21 18:17:18 0 [Note] Plugin 'FEEDBACK' is disabled. 2023-06-21 18:17:18 0 [ERROR] Unknown/unsupported storage engine: InnoDB 2023-06-21 18:17:18 0 [ERROR] Aborting

asked 10 months ago335 views
1 Answer
1

From your description and error logs, it seems like the problem is with your MySQL database, more specifically with the InnoDB storage engine. The error messages indicate that there is a discrepancy between your database and the log files, which suggests that your database might be corrupt.

InnoDB has a built-in mechanism for recovery in such situations, which you can use by adding or changing a line in your my.cnf file. Here's how you can do it:

  1. SSH into your Lightsail instance.

  2. Open the MySQL configuration file. For Bitnami instances, the path would typically be /opt/bitnami/mysql/my.cnf. You can open it with a text editor like nano or vi:

    sudo nano /opt/bitnami/mysql/my.cnf
  3. Look for a line that starts with innodb_force_recovery and change its value to 1. If such a line does not exist, add it under the [mysqld] section:

    [mysqld]
    innodb_force_recovery = 1
  4. Save the changes and exit the text editor. If you're using nano, you can do this by pressing Ctrl+X, then Y to confirm that you want to save the changes, and finally Enter to confirm the file name.

  5. Restart the MySQL service:

    sudo /opt/bitnami/ctlscript.sh restart mysql

After this, try accessing your WordPress site again. If it's still not working, you can try increasing the innodb_force_recovery value up to 6. However, please note that the higher the recovery mode, the higher the risk of losing data. Therefore, it's recommended to keep a backup of your data.

You can read more in the MySQL documentation: Forcing InnoDB Recovery

If the answer is helpful, please click "Accept Answer" and upvote it.

profile picture
EXPERT
answered 10 months ago
  • I did go with force recovery all the way to 5, but it did not fix my problem.

    When I try to login to database from the terminal, it show the following error: ERROR 2002 (HY000): Can't connect to local server through socket '/opt/bitnami/mariadb/tmp /mysql.sock' (111)

    any further guidance?

  • I did try restarting via ctlscript.sh multiple times. I allows me to get in my database for a brief time. I see my wordpress tables are somehow read only (options table, posts table etc.). Then after about 5 mins, it shows the above error again.

  • some more info my database is mariadb and my my.cnf path is /opt/bitnami/mariadb/conf/my.cnf

    when I create an instance with the snapshot and do status on ctlscript.sh it shows: cannot find any running daemon to contact. If it is running, make sure you are pointing to the right pid file (/var/run/gonit.pid) then i try to start ctlscript.sh it shows: Job for bitnami.service failed because the control process exited with error code. See "systemctl status bitnami.service" and "journalctl -xe" for details.

    I also see that mariadb is not running. and i cannot start it. Apache and php-fpm starts fine

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions