Automatic timeout on long running queries on AWS DocumentDB

Hello Sachin,

To manage long-running queries in AWS DocumentDB and prevent CPU spikes, try:

1. Server-Side Timeout Configuration: AWS DocumentDB itself does not provide a built-in server-side timeout setting for queries. You need to handle query timeouts within your application code using the maxTimeMS parameter provided by MongoDB drivers.

2. Using maxTimeMS for Query Timeouts: The maxTimeMS parameter allows you to specify a time limit for query execution. When the specified time is exceeded, the query will be terminated.

   Example:

   FindIterable<Document> result = collection.find().maxTime(1, TimeUnit.SECONDS);

3. Handling Parallel Queries: maxTimeMS applies to each query individually. It ensures that no single query runs longer than the specified time, which helps prevent CPU spikes caused by long-running queries. However, it does not directly manage the overall load from parallel queries.

   To mitigate spikes due to parallel queries:

   • Connection Pooling: Tune the connection pool settings to limit the number of concurrent operations.
   • Rate Limiting: Implement rate limiting in your application to control the number of queries being sent to DocumentDB.
   • Monitoring and Alerts: Use AWS CloudWatch to monitor CPU usage and set up alerts to take action when thresholds are exceeded.

By using maxTimeMS and managing query load through connection pooling and rate limiting, you can effectively control long-running queries and reduce CPU spikes.

I'm here to help!

ref.: https://docs.aws.amazon.com/documentdb/latest/developerguide/functional-differences.html

ref2: https://repost.aws/questions/QUEbaXMcc_T2O78Ls7b2_r1w/using-documentdb-with-lambda-connection-pooling

ref3: https://repost.aws/knowledge-center/documentdb-troubleshoot-high-cpu

Hi Sachin,

Here are some thoughts:

Configuring timeout for long-running queries in Amazon DocumentDB:
    Amazon DocumentDB does not have a built-in server-side mechanism to automatically timeout long-running queries.
    However, you can use the
    maxTimeMS
    option when executing queries to set a time limit. This will cause the query to be terminated if it exceeds the specified time limit.
    For example, in the MongoDB shell you can use the following command to set a 60-second timeout:

    db.collection.find({}).maxTimeMS(60000)

    This timeout setting applies to individual queries and does not provide a way to control spikes caused by parallel queries.

Handling spikes caused by parallel queries:
    Amazon DocumentDB does not have a built-in mechanism to automatically manage spikes caused by parallel long-running queries.
    To handle such spikes, you would need to implement custom monitoring and throttling mechanisms:
        Monitor the performance of your Amazon DocumentDB cluster using CloudWatch metrics and logs.
        Identify the queries causing performance issues and optimize them.
        Consider implementing client-side throttling to limit the number of concurrent queries, especially for long-running or resource-intensive operations.
        Explore the use of connection pooling or connection management strategies to better manage the load on your Amazon DocumentDB cluster.

https://docs.aws.amazon.com/documentdb/latest/developerguide/functional-differences.html#functional-differences.cursormaxTimeMS

https://repost.aws/knowledge-center/documentdb-slow-queries