Athena EXCEEDED_RESERVED_CAPACITY_LIMIT_EXCEPTION

0

I'm trying provisioned capacity in Amazon Athena with 24 or even 96 DPUs reserved, and even small queries are failing with the error:

EXCEEDED_RESERVED_CAPACITY_LIMIT_EXCEPTION: Requestedcount is greater than the limit

Simple query example:

SELECT
    *
FROM db.test
LIMIT 100

Table format: Iceberg.
Queries are being run through a specific workgroup ( Athena SQL engine version 3) that is linked to the reservation.

Any ideas on what could be misconfigured? Is there something else that needs to be done to ensure the workgroup actually uses the reserved DPUs?

asked 23 days ago45 views
1 Answer
-1

The error you're encountering suggests that there might be a mismatch between your query requirements and the capacity reservation configuration. Even though you've reserved 24 or 96 DPUs, the issue could be related to how the workgroup is configured or how the capacity is being utilized. Here are some potential reasons and solutions to consider:

  1. Query Requirements: Even simple queries might require more DPUs than expected, especially if the underlying data is large or complex. Athena determines the number of DPUs required for a query when it's submitted, and this can range from 4 to 124 DPUs for DML queries.

  2. Workgroup Configuration: Ensure that your workgroup is correctly linked to the capacity reservation. Double-check the workgroup settings to confirm it's using the reserved capacity.

  3. Concurrent Queries: If multiple queries are running concurrently, they might be consuming the reserved capacity. Check for other active queries that might be using up the DPUs.

  4. Capacity Allocation: Make sure the capacity is properly allocated to the workgroup. You might need to adjust the capacity allocation if multiple workgroups are sharing the same reservation.

  5. Query Queuing: If many queries are queued, it could indicate that your capacity is fully utilized. Monitor the Athena query queue time CloudWatch metric for your workgroup to check for excessive queuing.

To resolve this issue:

  1. Review your workgroup configuration and ensure it's correctly linked to the capacity reservation.

  2. Monitor your query concurrency and consider adding more DPUs to your reservation if needed.

  3. Check the CloudWatch metrics for your workgroup to identify any capacity bottlenecks.

  4. If the issue persists, you might need to contact AWS support to review your account's specific configuration and ensure there are no underlying issues with your capacity reservation setup.

Remember, increasing the number of DPUs in your reservation should help with query concurrency, but it doesn't necessarily mean that individual queries will have access to more resources. Each query still has its own resource requirements determined by Athena.
Sources
Determine capacity requirements - Amazon Athena
Resolve service limit or service quota errors in Athena | AWS re:Post
Resolve the TooManyRequestsException error in Athena | AWS re:Post

profile picture
answered 23 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions