Using concurrent write to DynamoDB

0

We have an api for sending SMS, in which we are saving a log in DynamoDB with the information about sending the SMS. We were testing the api and got the following results:

  • First Test: 10 requests are sent to the api, the api responds correctly to all requests, in CloudWatch the 10 calls are evident but in DynamoDB we only have 8 records.
  • Second Test: Another 10 requests are sent to the api with a different message, the api responds correctly to all the requests, in CloudWatch the 10 calls are evident but in DynamoDB we have 8 new records with the message of the second test and the two records that were missing from the first test. We did tests with a larger number of requests and the behavior is similar.

Does anybody have idea of ​​what's going on?

asked a year ago1593 views
1 Answer
1

Are you overwriting items in your DynamoDB table. Each item in DynamoDB must be uniquely identified by primary (Partition + Sort) key. For that reason, if more than 1 of your items share the same primary key, it would give the impression that the write did not occur, as it has just overwritten an existing item.

Some things to try:

  1. Make sure all items are unique
  2. Add a timestamp to your items to ensure they were updated/written when you expected them to be
  3. Enable Cloudtrail Dataplane Logs to see which data modifications you are making.
  4. Share a sample of the 10 items, along with the code you use to store them in DynamoDB. I can best help when I have more information.
profile pictureAWS
EXPERT
answered a year ago
  • We are using an email as Partition Key and something with this format YYYY-MM-DDTHH-mm-ss.mmss_SMS_UUID as Sort Key, so we can make sure that the Primary Key is unique and avoid overwriting.

    This is the code we are using to perform the tests, in each test we only change the SMS quantity and the SMS Message

    const axios = require("axios");

    let params = { message: " ", phoneNumber: "+573XXXXXXXXX", //This must be a mobile phone number }; const list = new Array(10); for (let index = 0; index < list.length; index++) { params.message = "test 10 sms: " + (index + 1); list[index] = JSON.stringify(params); } list.map(async (data) => { const apiKey = "<API KEY>"; var config = { method: "post", url: "https://yntwygrb91.execute-api.us-east-1.amazonaws.com/dev/sms", headers: { "Content-Type": "application/json", Accept: "application/json", "x-api-key": apiKey, }, data: data, }; axios(config) .then(function (response) { console.log(JSON.stringify(response.data)); }) .catch(function (error) { console.log(error); }); });

    And this is the code of the function that writes in DynamoDB

    exports.createLog = async function (logInfo, TableName) { const params = { TableName, Item: logInfo, }; try { const data = await dynamo.put(params).promise(); log(data); return data; } catch (err) { console.error(err); throw err; } };

  • Then please enable Dataplane Logging and see what is happening on the serverside.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions