- Newest
- Most votes
- Most comments
Yes, it is possible to update a list of objects in DynamoDB using Step Functions without writing a custom Lambda function.
The key is to use DynamoDB's list_append update expression to add the new list to the existing list attribute.
Here is an example state machine definition:
{ "StartAt": "UpdateDynamoDB", "States": { "UpdateDynamoDB": { "Type": "Task", "Resource": "arn:aws:states:::dynamodb:updateItem", "Parameters": { "TableName": "myTable", "Key": { "id": { "S": "123" } }, "UpdateExpression": "SET myList = list_append(myList, :newList)", "ExpressionAttributeValues": { ":newList": { "L": [ { "M": { "varA": { "S": "aaa" }, "varB": { "S": "bbb" } } }, { "M": { "varA": { "S": "aaaa"}, "varB": { "S": "bbbb" } } } ] } } }, "End": true } } }
The key points:
- Use
list_append
to add new list to existing list - Pass new list as an ExpressionAttributeValue
- Use DynamoDB's list data type for the attribute value
This will handle lists of any length.
To accomplish this , you use a map state and do updates for any single item as an iteration
{ "Type": "Map", "ItemsPath": "$.myList" "ItemProcessor": { "ProcessorConfig": { "Mode": "INLINE" }, "StartAt": "Update", "States": { "Update": { "Type": "Task", "Resource": "arn:aws:states:::dynamodb:updateItem", "Parameters": { "TableName": "TransferDataRecords-DDBTable-3I41R5L5EAGT", "Key": { "key": {"S.$": "$. varA"}, }, "UpdateExpression": "....", .... }, "End": true } } }, "End": true, }
Being aware of Map Execution limits can be useful , if you have a lot of items so probably you get the event history error reaching 25K limit, in this case you can choose the distributed map to by pas that limit as well optimise your process latency with the help of native parallelism power.
Another option can be to using a distributed map and for any child workflow execution use a map this way you have a Paralelized operation and yet you benefit from Map Iterative functionality.
To summerize I recommend you to have a look at following references Map State: https://docs.aws.amazon.com/step-functions/latest/dg/concepts-map-process-modes.html
Service Integrations: https://docs.aws.amazon.com/step-functions/latest/dg/connect-ddb.html
Hope this could be helpful
Thanks for the answer! Gave this a go and since i'm appending to a list in the database, this is leading to consistency issues where not all items are being updated in some cases. And this happens with all modes of map-run.
Due to this, decided to go ahead with updating a list of string (serializing the objects).
Relevant content
- asked a year ago
- asked 4 years ago
- asked 6 months ago
- asked 2 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
Thanks for the answer! As mentioned above,
myList
is of dynamic size (not always 2). Want to know if that can be achieved.