DynamoDB Index deployment via cloudformation issue: How to Deploy DynamoDB Indexes and Autoscaling via cloudFormation without deleting existing manually created indexes

0

We are currently facing an issue while deploying Global Secondary Index (GSI) autoscaling configurations for our DynamoDB table using AWS CloudFormation. The scenario is as follows:

  • In our production environment, certain GSIs were manually created along with their respective autoscaling configurations.
  • To maintain code consistency and eliminate manual interventions, we now intend to deploy the same GSIs and autoscaling configurations using CloudFormation.
  • However, during the CloudFormation deployment, we encounter the following error: Error Message:
The following resource(s) failed to create: [------GSIReadScalableTarget,--------GSIWriteScalableTarget]. The following resource(s) failed to update: [----------IndexWriteScalingPolicy, -----IndexReadScalingPolicy].

Our key concern is whether there is a provision to deploy these indexes and autoscaling configurations via CloudFormation without requiring the deletion of the existing manually created indexes. We would appreciate your guidance on:

  • The correct approach to update the manually created GSIs and their autoscaling policies using CloudFormation.
  • Whether CloudFormation supports adopting existing GSIs and their autoscaling policies without recreating them.
  • Any best practices or workarounds to avoid disruptions in our production environment while achieving the desired configuration. Looking forward to your assistance.
asked 11 days ago48 views
2 Answers
1

Deploying DynamoDB Global Secondary Indexes (GSIs) and their autoscaling configurations via CloudFormation when they already exist manually can indeed be challenging. Here's how you can approach this situation:

  1. Existing GSIs: CloudFormation can manage existing GSIs without recreating them. You can include the existing GSI definitions in your CloudFormation template, and CloudFormation will recognize them as already existing resources. This way, you won't need to delete the manually created indexes.

  2. Autoscaling Configurations: The error message you're encountering suggests that the issue is primarily with the autoscaling configurations rather than the GSIs themselves. CloudFormation doesn't have a direct way to "adopt" existing autoscaling configurations. You'll need to define these in your CloudFormation template.

  3. Deployment Strategy: a. First, include the existing GSI definitions in your CloudFormation template without any changes to their structure. b. Then, add the autoscaling configurations for these GSIs in the same template. c. When you deploy, CloudFormation should recognize the existing GSIs and attempt to add the new autoscaling configurations.

  4. Potential Issues: If the autoscaling resources (like ScalableTargets and ScalingPolicies) already exist due to manual creation, CloudFormation might fail to create them as new resources. In this case, you have two options: a. Temporarily remove the existing autoscaling configurations manually before deploying the CloudFormation stack. b. Use a custom resource in CloudFormation to check if the autoscaling configurations exist and update them instead of creating new ones.

  5. Best Practices: a. Always test your CloudFormation changes in a non-production environment first. b. Use CloudFormation drift detection to identify any manual changes made outside of CloudFormation. c. Consider using the DynamoDB UpdateTable API through a custom resource if you need more fine-grained control over the update process.

  6. Gradual Approach: If you're concerned about potential disruptions, consider a gradual approach: a. Start by including just the GSI definitions in CloudFormation without autoscaling. b. Once that's successful, add autoscaling for one GSI at a time. c. Monitor closely and be prepared to rollback if issues occur.

Remember, while CloudFormation is updating the table, it doesn't wait for index creation to complete because the backfilling phase can take a long time. You can track the index status using the DynamoDB DescribeTable command.

By following these steps and best practices, you should be able to bring your manually created GSIs and their autoscaling configurations under CloudFormation management without deleting existing indexes or causing significant disruptions to your production environment.
Sources
AWS::DynamoDB::Table - AWS CloudFormation
AWS::DynamoDB::GlobalTable - AWS CloudFormation
Amazon DynamoDB template snippets - AWS CloudFormation

profile picture
answered 11 days ago
0

My suggestion here would be to convert the table to on-demand mode. This will remove any autoscaling settings. Then add the GSI's to the CFN template and it should complete. Then flip back to provisioned with your autoscaling settings in place. Of course, this needs tested thoroughly in a lower environment.

profile pictureAWS
EXPERT
answered 11 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions