Creating a DynamoDB Table with CloudFormation and Adding Items at Creation Time

4 minute read
Content level: Intermediate
1

How to create a DynamoDB table using CloudFormation while adding items to it at creation time.

Amazon DynamoDB is a fully managed NoSQL database service provided by AWS, known for its scalability, low latency, and seamless integration with other AWS services. CloudFormation, AWS's infrastructure-as-code service, allows you to define and provision resources in a declarative manner. In this article, we'll explore how to create a DynamoDB table using CloudFormation and add items to it at the time of creation.

Prerequisites

Before we begin, make sure you have the following prerequisites in place:

  • An AWS account with appropriate permissions to create resources using CloudFormation.
  • Familiarity with AWS CloudFormation and basic knowledge of AWS services, including DynamoDB and Lambda.
  • An understanding of YAML or JSON, as CloudFormation templates can be written in either format.

Step 1: Writing the CloudFormation Template

Let's start by creating the CloudFormation template to define our DynamoDB table and a Lambda function that will add items to the table. The CloudFormation template is written in YAML format for simplicity.

AWSTemplateFormatVersion: '2010-09-09'
Resources:
  MyDynamoDBTable:
    Type: 'AWS::DynamoDB::Table'
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: pk
          AttributeType: S
        - AttributeName: sk
          AttributeType: S
      KeySchema:
        - AttributeName: pk
          KeyType: HASH
        - AttributeName: sk
          KeyType: RANGE
  MyCustomResourceLambdaFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      Runtime: nodejs14.x
      Handler: index.handler
      Role: !GetAtt MyLambdaExecutionRole.Arn
      Environment:
        Variables:
          tableName: !Ref MyDynamoDBTable
      Code:
        ZipFile: |
          const AWS = require('aws-sdk');
          const response = require('cfn-response');
          const client = new AWS.DynamoDB();
          const dynamodb = new AWS.DynamoDB.DocumentClient();
     
          exports.handler = async (event, context) => {
            try {
              const tableName = process.env.tableName
              console.log(tableName)
              var params = {
                'TableName': tableName
              };
            
              client.waitFor('tableExists', params, function(err, data) {
                if (err) console.log(err, err.stack); // an error occurred
                else     console.log(data);           // successful response
              });

              const itemsToAdd = [
                { pk: 'item1', sk: 'sortKey1', otherAttribute: 'value1' },
                { pk: 'item2', sk: 'sortKey2', otherAttribute: 'value2' },
                { pk: 'item3', sk: 'sortKey3', otherAttribute: 'value3' },
                { pk: 'item4', sk: 'sortKey4', otherAttribute: 'value4' },
                { pk: 'item5', sk: 'sortKey5', otherAttribute: 'value5' },
              ];
              
              const putItemPromises = itemsToAdd.map((item) => {
                const params = {
                  TableName: tableName,
                  Item: item,
                };
                return dynamodb.put(params).promise();
              });
              
              await Promise.all(putItemPromises).then(res=>console.log(res)).catch(err=>console.log(err))
              
              const responseData = { Result: 'Items added successfully' };
              await response.send(event, context, response.SUCCESS, responseData);
            } catch (error) {
              console.log(error)
              const responseData = { Error: 'Something went wrong' };
              await response.send(event, context, response.FAILED, responseData);
            }
          };
      Timeout: 30
  MyLambdaExecutionRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
        - 'arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess'
  MyCustomResource:
    Type: 'Custom::MyCustomResource'
    Properties:
      ServiceToken: !GetAtt MyCustomResourceLambdaFunction.Arn

Step 2: Understanding the CloudFormation Template

In this CloudFormation template, we define two main resources:

  • MyDynamoDBTable: This resource creates the DynamoDB table with the required schema. It sets the billing mode to PAY_PER_REQUEST, which means that you pay only for the read and write capacity that your application consumes.
  • MyCustomResourceLambdaFunction: This resource creates a custom Lambda function backed by a CloudFormation Custom Resource. The Lambda function is responsible for adding five sample items to the DynamoDB table. We use waitFor('tableExists') to ensure that the Lambda function waits until the DynamoDB table is fully created or updated before adding the items.

Step 3: Deploying the CloudFormation Stack

To deploy the CloudFormation stack, follow these steps:

  1. Save the CloudFormation template in a YAML file (e.g., dynamodb-stack.yaml).
  2. Open the AWS Management Console, navigate to the CloudFormation service, and click on "Create stack."
  3. Select "Upload a template file," and then upload the dynamodb-stack.yaml file you saved in the previous step.
  4. Provide a stack name and any additional parameters required by your template.
  5. Review the configuration and click "Create stack" to initiate the deployment.

Using the CLI:

aws cloudformation create-stack \
--stack-name <stack-name> \
--template-body file://myfile.yaml \
--capabilities CAPABILITY_NAMED_IAM

Step 4: Observing the Result

After the CloudFormation stack is created, you can navigate to the DynamoDB console and find the table which has a name that begins with the name of your CloudFormation stack. You will notice that the table has been created and contains five items with the specified partition key pk and sort key sk.

Conclusion

In this article, we explored how to use AWS CloudFormation to create a DynamoDB table with a custom schema and provision sample items at the time of table creation. By leveraging the power of CloudFormation and custom Lambda resources, you can seamlessly manage your AWS infrastructure and automate the process of adding initial data to your DynamoDB tables. This approach ensures that your DynamoDB tables are always initialized with the required data, providing a solid foundation for your applications to build upon.

profile pictureAWS
EXPERT
published a year ago5976 views
3 Comments

Hello Lee,

It works but actually after stack creation , during creation of MyCustomResource it was taking so much of time as it is not getting the response back and then create for custom resource failed with the reason "CloudFormation did not receive a response from your Custom Resource" and then due to rollback everything got deleted. Could you please suggest something and help me to resolve this issue as well ?

Thanks

replied a year ago

Thanks for the update, there was an await missing for the response.send() request.

profile pictureAWS
EXPERT
replied a year ago

Ingestion of data during the table creation is a classic example, but by doing it via Lambda, we are keeping an one-off Lambda in our Account, We need to write another script to decommission the Lambda, if decision is to leave the Lambda, we need to take care, if some one accidentally runs lambda , don't ingest duplicate data into Dynamo Table. My opinion, from where the CLI for Cloud-formation is being run, excute node.js code (or some other tech) to ingest data into DynamoDB table

Pavan
replied 10 months ago