RDS ACUUtilization 100% even without any load. The Proxy has 0 incoming client connections. Even the number of database connections is 0 on Proxy monitoring.
Dear Team,
RDS ACUUtilization 100% even without any load. The Proxy has 0 incoming client connections. Even the number of database connections is 0 on Proxy monitoring.
But when I check on the performance insights in RDS, rdsproxyadmin is connecting and for godsake or AWS sake constantly running the below query.
SELECT current_setting(?) AS transaction_read_only, current_setting(?) AS max_connections, EXTRACT(? FROM now() - pg_postmaster_start_time()) AS uptime, durable_lsn, current_read_lsn FROM aurora_replica_status() WHERE server_id = ? ORDER BY (session_id = ?) ASC LIMIT ?
Before I enabled the proxy all was fine. I had a DB load from 09:00 to 16:00 and by 16:30 DBs cooled down, and the ACUUtilization back to 0 and the costs were OK, now with out any load on DB I am getting day robbed, 3 times the cost on db as before.
Is there a way to schedule Proxy or stop this unnecessary behaviour ??
Definitely this is not acceptable to me, If I am not have tangible solution, I am forced to find alternatives with other service providers.
Thank you.
- Newest
- Most votes
- Most comments
I understand your frustration with the unexpected behavior and increased costs you're experiencing after enabling RDS Proxy. The situation you're describing is indeed concerning, and I'll try to provide some insights and potential solutions.
The constant query you're seeing from rdsproxyadmin is part of RDS Proxy's health check mechanism. It's designed to maintain awareness of the database's status and ensure quick failover if needed. However, it shouldn't be causing such a significant increase in ACU utilization or costs under normal circumstances.
Here are some steps you can take to address this issue:
-
Check your RDS Proxy configuration:
- Review the connection pool settings, particularly the MaxConnectionsPercent and MaxIdleConnectionsPercent. You might need to adjust these to better suit your workload.
- Ensure that the proxy is configured with the correct target group and database instance.
-
Examine your database instance configuration:
- Verify that the instance size is appropriate for your workload.
- Check if any recent changes in database parameters might be affecting performance.
-
Investigate potential query optimization:
- Although the query you're seeing is a system query, its frequent execution might indicate an underlying issue. Consider reviewing your database's query performance and optimization.
-
Monitor and analyze metrics:
- Continue to monitor CloudWatch metrics for both the RDS instance and the RDS Proxy to identify any patterns or anomalies.
-
Consider using RDS Proxy scheduling:
- While there isn't a built-in scheduling feature for RDS Proxy, you could implement a custom solution using AWS Lambda and CloudWatch Events to start and stop the proxy based on your usage patterns.
-
Engage AWS Support:
- Given the severity of the issue and its impact on your costs, it would be advisable to open a support ticket with AWS. They can provide more detailed analysis and potentially identify any misconfigurations or bugs.
If these steps don't resolve the issue, you might consider temporarily disabling the RDS Proxy while further investigating the root cause. However, be aware that this will remove the benefits that RDS Proxy provides, such as connection pooling and reduced failover times.
It's important to note that while RDS Proxy can provide significant benefits in many scenarios, it may not be the best fit for all use cases. If after thorough investigation and optimization you find that it's not providing value for your specific workload, it's reasonable to consider alternative solutions.
Remember, AWS is generally responsive to customer feedback and issues. If this is indeed a bug or unintended behavior, reporting it could lead to improvements that benefit all users.
Sources
modify-db-proxy-target-group — AWS CLI 2.24.7 Command Reference
register-db-proxy-targets — AWS CLI 2.24.7 Command Reference
delete-db-proxy-endpoint — AWS CLI 2.24.7 Command Reference
The Point 5 (Consider using RDS Proxy scheduling:) Sounds interesting to me, others are general, because as i said the DB has zero incoming load and many time the CPU% is hardly 60%. Do you have the command to schedule or any documentation to do RDS proxy scheduling with Lambda? Thank you.