How can I troubleshoot index creation failures in my Amazon DocumentDB cluster?

Short description

Indexes are B-tree structures used to improve query performance in Amazon DocumentDB. When you use indexes, you avoid the need for a collection scan operation that takes more time and consumes more DB resources. For more information on the types of indexes and their use cases, see How to index on Amazon DocumentDB (with MongoDB compatibility).

Amazon DocumentDB allows you to create only one index at a time at the collection level. But you can use a parallel index update to make parallel the data loading into collection. It's a best practice to create indexes first and then start loading the data.

Amazon DocumentDB supports the following two methods of index creation as overview:

• Background index creation allows other database operations to run in the foreground while the index creates.
• Foreground index creation blocks all other database operations until the index is created. This is generally a faster method.

Resolution

Use the following steps to troubleshoot index creation failures in your Amazon DocumentDB cluster:

Resolve "Command failed with error 14031: 'Not enough disk space' on server" errors

Amazon Document uses a DB instance's available FreeLocalStorage to create an index based on the instance class of the primary resource. The storage size needed depends on the size of the index that's created. If the FreeLocalStorages value drops to a low value during index creation, then the index creation can slow down. Then an error similar to the following is thrown:

"Error message: Command failed with error 14031: 'Not enough disk space' on server."

To monitor the available storage, set up an Amazon CloudWatch alarm on the FreeLocalStorage metric. This allows you to take proactive action when the available storage is low. If you create an index on a large collection, then it's a best practice to scale up the instance class before you create the index. Then scale back after the index creation is complete.

Resolve "Existing index build in progress on the same collection. Collection is limited to a single index build at a time" errors

Amazon DocumentDB supports only one background index creation at a time per collection. For example, you might try to run DDL operations such as createIndex() or you might dropIndex() more than one index on a given collection. In this use case, the background index build fails with an error similar to the following:

"Error message: "Existing index build in progress on the same collection. Collection is limited to a single index build at a time".

To avoid this error, make sure to run only one background index creation at a time per collection. Run the following command to review your ongoing index creations:

db.currentOp({"command.createIndexes": { $exists : true } })

Resolve "Key too large to index" errors

If you create an index with a key size greater than 2048 bytes, then index creation fails with the "key too large to index" error. For more information on key size limits, see Amazon DocumentDB quotas and limits.

Resolve "createIndex error: namespace name generated from index name is too long" errors

You might try to migrate from MongoDB to Amazon DocumentDB with an index name ($) that exceeds 63 characters. In this use case, the index creation fails with following error:

Error message: "createIndex error: namespace name generated from index name is too long"

Review the Amazon DocumentDB naming constraints to address this error.

Resolve issues caused by throttling

Throttling in your DB instance resources can cause index operations to fail. Before you create an index, make sure that there's no throttling present in CPU, memory, DBConnections, and FreeLocalStorage. Use CloudWatch graphs, Performance Insights, and Enhanced Monitoring tools to validate the load before index creation.

Resolve issues caused by long running transactions

Long running transactions can cause your index creation to fail. Make sure to perform live troubleshooting to understand why index creation is taking a long time and what the wait type is.

Use the following query to check which transactions are currently in progress:

db.currentOp( { "active": true, $and: \[ { op: "command", "command.createIndexes": { $exists: true } } \] } )

Resolve "createIndex error: Field 'weights' is currently not supported" errors

If you try to restore data from MongoDB and you use a text type index, then your index creation fails with the following error:

"Error message: createIndex error: Field 'weights' is currently not supported".

The text index type isn't supported for any of the current Amazon DocumentDB versions. To resolve this issue, drop the index or change the index type in the source server. Then take a fresh data dump and restore it to AWS.

Resolve "unsupported_index_options" errors

Amazon DocumentDB doesn't support the creation of partial indexes or 2dsphere. When you try to create a partial index or 2dsphere, you get an error similar to the following:

"Error message: "unsupported_index_options": [ "partialFilterExpression" ] or ["2dsphereIndexVersion"]"

To avoid migration failures caused by unsupported index types, change the index format to a supported type, and then re-initiate the migration.

Related information

Index creation

How to index on Amazon DocumentDB (with MongoDB compatibility)

Amazon DocumentDB quotas and limits