AWS Aurora issues with MySQL.Data connector for .NET clients

0

With our upgrade of Aurora from 2.10.2 to 2.11 we've experienced connection timeouts to one of our DBs in our aurora clusters that couldn't be explained by load or number of connections. We've investigated and found that the MySQL.Data library for .Net as provided by Oracle appears to have issues with this latest version of Aurora which can cause the server to block new connections. We have reproduced this using the latest version of MySQL.Data 8.0.31 (as well as past versions of this package) and issuing parallel connections. The server ceases to accept new connections after two parallel connections are made. In error logs we see "Bad handshake" and "Got an error reading communication packets" errors and the user shows as "unauthenticated". When pointing at a cluster using 2.10.2 we cannot replicate any issues and the MySQL.Data library works as expected.

Are there any advisory's on this?

asked a year ago240 views
1 Answer
0

Hello,

The 'Got an error reading communication packets' is a generic warning message and could appear due to a variety of reasons including improper termination of a client connection, sleeping connections exceeding the 'wait_timeout' or the 'interactive_timeout' values and consequently being forcibly closed by the Aurora MySQL server, the client exceeding the 'max_allowed_packet' value for queries etc.

Some of the common reasons for such error messages are as follows:

  1. It takes more than connect_timeout seconds to obtain a connect packet.
  2. The client had been sleeping more than wait_timeout or interactive_timeout seconds without issuing any requests to the server. I can see that you have increased the time out values for these parameters.
  3. The max_allowed_packet variable value is too small or queries require more memory than you have allocated for mysqld.
  4. The client program did not call mysql_close() before exiting.
  5. The client program ended abruptly in the middle of a data transfer.
  6. A connection packet does not contain the right information.

These are some checks that could be made:

  • show full processlist;
  • SHOW GLOBAL STATUS like '%aborted%';
  • select * from innodb_locks;
  • select * from innodb_trx;
  • select * from innodb_lock_waits;
  • Show engine InnoDB Status

This will give you some extra information in order to dive into this issue since the information about the status of the connections is internal and due to security reasons we do not have access to this data.

Here are some possible remediation methods that might help you resolve/reduce the occurrence of these errors :

  1. Close the connection properly on the application side rather than waiting for it to timeout. We strongly recommend changing the application logic to properly close connections at the end of an operation, this can be done by calling the close() function from within your application.
  2. Set 'wait_timeout' to a higher value (max. value possible) [+] https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_wait_timeout
  3. Set max_allowed_packet to a higher value [+] https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_max_allowed_packet
  4. If you are using an application to connect, make sure the timeout value on the application end is not shorter than the 'wait_timeout' otherwise connections will be aborted abruptly from the application end thus causing this error.

You may also refer to the following links for more information:

• How do I resolve the "Got an error reading communication packets" error in Amazon RDS MySQL? : https://aws.amazon.com/premiumsupport/knowledge-center/rds-mysql-communication-packet-error/https://www.percona.com/blog/2016/05/16/mysql-got-an-error-reading-communication-packet-errors/
https://www.percona.com/blog/2008/08/23/how-to-track-down-the-source-of-aborted_connects/
https://aws.amazon.com/blogs/database/best-practices-for-configuring-parameters-for-amazon-rds-for-mysql-part-3-parameters-related-to-security-operational-manageability-and-connectivity-timeout/
• Aurora MySQL database engine updates 2022-10-25 (version 2.11.0) : https://docs.aws.amazon.com/AmazonRDS/latest/AuroraMySQLReleaseNotes/AuroraMySQL.Updates.2110.html • MySQL bugs fixed by Aurora MySQL database engine updates : https://docs.aws.amazon.com/AmazonRDS/latest/AuroraMySQLReleaseNotes/AuroraMySQL.Updates.MySQLBugs.html

I hope the above information helps!

Thank you and wishing you a great day ahead!

AWS
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions