RDS Proxy after upgrade

0

Earlier this year I upgraded one of our test instances of RDS MySQL from version 5.7 to version 8.0. We'd been using RDS Proxy without issue but after the upgrade it wasn't possible to connect to MySQL via proxy with the error reported as:

ERROR 1105 (HY000): Unknown error

At the time I didn't have chance to investigate more but I did try deleting the proxy and creating a new one and after that we could again connect to MySQL via RDS Proxy. I've now come back to this and would prefer to not have to deleted and create a new Proxy after the upgrade so wondered if anyone else has had this experience or know if this might have been a bug that has been since fixed?

Many thanks in advance

1 Answer
0

1105 is a generic error that occurs when attempting to connect to a MySQL proxy [1]. When connecting an RDS proxy to a DB instance, this is not the expected behavior. The following factors contribute to RDS Proxy connection failures [2]:

  1. The connection is being blocked by security group rules, either at the DB instance or at the RDS Proxy.
  2. RDS proxy only works within a VPC, therefore, connecting from outside the private network will fail.
  3. When using native user name/password mode, the authentication credentials are incorrect and cause the connection to fail.
  4. The client's AWS Identity and Access Management (IAM) user or role is not authorized to connect to RDS Proxy (when IAM DB authentication is used).

Please confirm the following [3]:

1] The role being attached to the proxy has the below policy. { "Version": "2012-10-17", "Statement": [ { "Sid": "GetSecretValue", "Action": [ "secretsmanager:GetSecretValue" ], "Effect": "Allow", "Resource": "*" --->arn of secrets }, { "Sid": "DecryptSecretValue", "Action": [ "kms:Decrypt" ], "Effect": "Allow", "Resource": [ "arn of kms" --->This is in case secrets is encrypted ] } ] }

2] Please check the database username and password in AWS Secrets Manager and try connecting to the username and password to ensure the details in the secrets manager are correct.

3] The RDS security group has an inbound rule from the proxy security group and an outbound rule to 0.0.0.0. (or outbound to proxy security). The proxy security group must communicate with the RDS database.

Additionally, please answer the following questions regarding the behavior of your RDS proxy and DB instance to further assist in this investigation.

  1. Is the error displayed after the connection has been established or while it is attempting to connect the proxy to the DB instance?
  2. Is the problem present when connecting to the DB instance without the proxy?

Resources:

[1] https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy.troubleshooting.html

[2] https://aws.amazon.com/premiumsupport/knowledge-center/rds-proxy-connection-issues/

[3] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy-setup.html#rds-proxy-network-prereqs

AWS
SUPPORT ENGINEER
answered 6 months ago
  • I really appreciate you getting back to me.

    As I mentioned in my original post the RDS Proxy and RDS MySQL were both working and we were able to connect to them before the upgrade from both the mysql command line as well as from our applications via JDBC. The only thing that changed was that I changed the mysql version in our CloudFormation template which caused the MySQL Instance to be upgraded.

    On the points you asked me to confirm, I can I indeed confirm that they were all correct before and after the upgrade however after the upgrade connecting from either mysql command line or via our applications always gave us "ERROR 1105 (HY000): Unknown error" when connecting via the proxy but we could connect directly just fine. In order to fix the issue I removed the AWS::RDS::DBProxy definition from the CloudFormation template, updated the stack and then put the definition back in and finally updated the stack again. Same passwords, same security groups and role policy, everything was identical. It was as if the RDS Proxy was somehow tied to the version of MySQL and that something was incompatible once the MySQL instance had been upgraded.

    I could do the same for our production environment however we'd rather not suffer more downtime than we have to so not having to replace it would be better, so it would be good to understand if this is still an issue.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions