High DatabaseConnectionsBorrowLatency spike on RDS Proxy but reasonable client connection count

What database are you using? I had slower performance when I enabled RDS proxy with Sql Server and received suggestions from AWS not to use RDS proxy with Sql Server but instead to rely on Sql Server's capability directly.

The high DatabaseConnectionsBorrowLatency you're experiencing with RDS Proxy, despite low database load and reasonable connection counts, could be caused by several factors.

This metric specifically measures the time it takes for RDS Proxy to acquire a database connection from the connection pool. A high value (5000ms+) indicates that your proxy is struggling to provide connections quickly, even though you're not seeing a connection storm.

Possible causes and solutions:

1. Maximum connections limit: Even if you're not seeing a connection storm, you might be reaching the maximum allowed connections to your database. RDS Proxy sets this limit based on the MaxConnectionsPercent setting (a percentage of your database's max_connections parameter). Check if you're approaching this limit by comparing the DatabaseConnections and MaxDatabaseConnectionsAllowed metrics.

2. ConnectionBorrowTimeout setting: This determines how long RDS Proxy waits for a database connection to become available before timing out. Your current setting might be too high, causing long waits instead of failing fast. Consider reviewing and adjusting this value.

3. Connection pinning: If your application is causing connection pinning (where RDS Proxy can't reuse connections across client sessions), this reduces the efficiency of connection pooling. Certain operations like using temporary tables, SET commands, or multi-statement transactions can cause pinning. Review your application code for these patterns.

4. Database availability issues: The database might be temporarily unavailable or slow to establish connections, even if it appears to be under low load.

5. Proxy target group configuration: Check if your target group is properly configured and the database is available. The AvailabilityPercentage CloudWatch metric can help determine this.

To troubleshoot:

• Review your ConnectionBorrowTimeout and MaxConnectionsPercent settings
• Check for connection pinning in your application
• Monitor the DatabaseConnections and MaxDatabaseConnectionsAllowed metrics
• Examine the database's availability metrics

If the issue persists after these checks, you might need to contact AWS Support for further investigation.
Sources
RDS Proxy connection considerations - Amazon Relational Database Service
Troubleshooting for RDS Proxy - Amazon Relational Database Service
Troubleshooting for RDS Proxy - Amazon Aurora
RDS Proxy connection considerations - Amazon Aurora