StepFunctions update list of objects into DynamoDB

Question

I have a DynamoUpdateItem taskState in my state-machine in which I want to update a list of objects into the dynamodb table. The input to the step looks something like: 
```
{
    "myList": [
        {
            "varA": "aaa",
            "varB": "bbb"
        },
        {
            "varA": "aaaa",
            "varB": "bbbb"
        },
        ...
    ]
}
```
I want to update `myList` as is into my DDB entry. Note that the size of `myList` for each execution is not static.

So far, i've tried using `DynamoAttributeValue.listFromJsonPath` but that doesn't work as SFN does not recognize the individual objects (map) within the list provided. Is there a way this requirement can be achieved without having to write a custom lambda function?

Answer

Yes, it is possible to update a list of objects in DynamoDB using Step Functions without writing a custom Lambda function.

The key is to use DynamoDB's list_append update expression to add the new list to the existing list attribute.

Here is an example state machine definition:

```json
{
  "StartAt": "UpdateDynamoDB",
  "States": {
    "UpdateDynamoDB": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:updateItem",
      "Parameters": {
        "TableName": "myTable",
        "Key": {
          "id": { "S": "123" } 
        },
        "UpdateExpression": "SET myList = list_append(myList, :newList)",
        "ExpressionAttributeValues": {
          ":newList": {
            "L": [
              {
                "M": {
                  "varA": { "S": "aaa" },
                  "varB": { "S": "bbb" }
                }
              },
              {
                "M": {
                  "varA": { "S": "aaaa"}, 
                  "varB": { "S": "bbbb" }
                }
              }
            ]  
          }
        }
      },
      "End": true
    }
  }
}
```

The key points:

- Use `list_append` to add new list to existing list
- Pass new list as an ExpressionAttributeValue
- Use DynamoDB's list data type for the attribute value

This will handle lists of any length.

Answer

To accomplish this , you use a map state and do updates for any single item as an iteration

```json
{
    "Type": "Map",
    "ItemsPath": "$.myList"
    "ItemProcessor": {
        "ProcessorConfig": {
            "Mode": "INLINE"
        },
        "StartAt": "Update",
        "States": {
            "Update": {
                "Type": "Task",
                 "Resource": "arn:aws:states:::dynamodb:updateItem",
                 "Parameters": {
                       "TableName": "TransferDataRecords-DDBTable-3I41R5L5EAGT",
                       "Key": {
                             "key": {"S.$": "$. varA"},
                         },
                        "UpdateExpression": "....",
                       ....
                 },
                "End": true
            }
        }
    },
    "End": true,
}
```

Being aware of Map Execution limits can be useful , if you have a lot of items so probably you get the event history error reaching 25K limit, in this case you can choose the distributed map to by pas that limit as well optimise your process latency with the help of native parallelism power.

Another option can be to using a distributed map and for any child workflow execution use a map this way you have a Paralelized operation and yet you benefit from Map Iterative functionality.

To summerize I recommend you to have a look at following references 
Map State:  https://docs.aws.amazon.com/step-functions/latest/dg/concepts-map-process-modes.html

Service Integrations: https://docs.aws.amazon.com/step-functions/latest/dg/connect-ddb.html

Hope this could be helpful

StepFunctions update list of objects into DynamoDB

相关内容