- Newest
- Most votes
- Most comments
Based on your error logs and scenario description, there are several potential causes for the temporary connection failure between your Fargate cluster in VPC-A and your RDS MySQL instance in VPC-B:
-
Connection Limit Issue: The error log shows "Too many connections" warnings, which indicates that your RDS instance reached its maximum allowed connections. This can happen if your application doesn't properly close connections or if there was a sudden spike in connection attempts.
-
DNS Resolution Problem: The log entry "IP address '172.31.8.72' could not be resolved: Name or service not known" suggests a DNS resolution issue. This could be temporary network connectivity problems between your VPCs.
-
RDS Maintenance: If AWS performed maintenance on your RDS instance during that time window, it could have caused a brief connectivity disruption. RDS maintenance can sometimes cause temporary connectivity issues, especially if it involved a failover or configuration changes.
-
Network Connectivity Issues: Since your Fargate cluster and RDS instance are in different VPCs, the connection depends on VPC peering, Transit Gateway, or other networking components. Any disruption in these components could cause temporary connectivity issues.
-
NAT Gateway Issues: While NAT gateway failure could be a cause, it's less likely if your RDS instance is public and your Fargate tasks are connecting directly to it. However, if your architecture uses a NAT gateway for this communication path, then a NAT gateway issue could indeed cause this problem.
To prevent future occurrences, I recommend:
- Monitoring your RDS connection count and increasing the max_connections parameter if needed
- Implementing connection pooling in your application
- Setting up CloudWatch alarms for RDS connectivity metrics
- Reviewing your network architecture to ensure resilience
- Checking AWS Health Dashboard for any service disruptions during that time period
The fact that the connection recovered on its own suggests a temporary issue rather than a configuration problem.
Sources
RDS connectivity failure after AWS maintenance window | AWS re:Post
Hello.
Since the error message "Too many connections" is recorded, I think the limit on the number of connections on the RDS side has been exceeded.
Check the RDS metric "DatabaseConnections" during the relevant time period to see if the number of connections has increased.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-metrics.html
If the number of connections to the database is increasing, increasing values such as max_connections in the RDS parameter group may solve the problem.
https://repost.aws/knowledge-center/rds-mysql-max-connections
Relevant content
asked 2 years ago
asked 2 years ago
asked 2 years ago
