How can I troubleshoot index creation failures in my Amazon DocumentDB cluster?
5 minute read
My Amazon DocumentDB (with MongoDB compatibility) DB cluster takes a long time to create an index. Or, I get an error when I try to create an index on my Amazon DocumentDB cluster.
Indexes are B-tree structures used to improve query performance in Amazon DocumentDB. When you use indexes, you avoid the need for a collection scan operation that takes more time and consumes more DB resources. For more information on the types of indexes and their use cases, see How to index on Amazon DocumentDB (with MongoDB compatibility).
Amazon DocumentDB allows you to create only one index at a time at the collection level. But you can use a parallel index update to make parallel the data loading into collection. It's a best practice to create indexes first and then start loading the data.
Background index creation allows other database operations to run in the foreground while the index creates.
Foreground index creation blocks all other database operations until the index is created. This is generally a faster method.
Use the following steps to troubleshoot index creation failures in your Amazon DocumentDB cluster:
Resolve "Command failed with error 14031: 'Not enough disk space' on server" errors
Amazon Document uses a DB instance's available FreeLocalStorage to create an index based on the instance class of the primary resource. The storage size needed depends on the size of the index that's created. If the FreeLocalStorages value drops to a low value during index creation, then the index creation can slow down. Then an error similar to the following is thrown:
"Error message: Command failed with error 14031: 'Not enough disk space' on server."
To monitor the available storage, set up an Amazon CloudWatch alarm on the FreeLocalStorage metric. This allows you to take proactive action when the available storage is low. If you create an index on a large collection, then it's a best practice to scale up the instance class before you create the index. Then scale back after the index creation is complete.
Resolve "Existing index build in progress on the same collection. Collection is limited to a single index build at a time" errors
Amazon DocumentDB supports only one background index creation at a time per collection. For example, you might try to run DDL operations such as createIndex() or you might dropIndex() more than one index on a given collection. In this use case, the background index build fails with an error similar to the following:
"Error message: "Existing index build in progress on the same collection. Collection is limited to a single index build at a time".
To avoid this error, make sure to run only one background index creation at a time per collection. Run the following command to review your ongoing index creations:
If you create an index with a key size greater than 2048 bytes, then index creation fails with the "key too large to index" error. For more information on key size limits, see Amazon DocumentDB quotas and limits.
Resolve "createIndex error: namespace name generated from index name is too long" errors
You might try to migrate from MongoDB to Amazon DocumentDB with an index name ($) that exceeds 63 characters. In this use case, the index creation fails with following error:
Error message: "createIndex error: namespace name generated from index name is too long"
Throttling in your DB instance resources can cause index operations to fail. Before you create an index, make sure that there's no throttling present in CPU, memory, DBConnections, and FreeLocalStorage. Use CloudWatch graphs, Performance Insights, and Enhanced Monitoring tools to validate the load before index creation.
Resolve issues caused by long running transactions
Long running transactions can cause your index creation to fail. Make sure to perform live troubleshooting to understand why index creation is taking a long time and what the wait type is.
Use the following query to check which transactions are currently in progress: