BatchWritteItem in DynamoDB Ignore Empty Values

Question

Hello All!

I have this challenge where I have an automation process consuming the API of BatchWritteItem. My JSON request is fixed to POST 4 items on each batch:

```
[
  {
    "salesforce_id": "00Q7V00001tCDJ6Uyrtbgtyun",
    "type": "Lead",
    "event_name": "New Lead",
    "Email": "sebasXXXX@gmail.com",
    "Phone": "[Object: null]",
    "MobilePhone": "+XXXX62701",
    "Name": "Sebastian Hoyos",
    "FirstName": "Sebastian",
    "LastName": "Hoyos",
    "City": "[Object: null]",
    "SBF_Pais__c": "Colombia",
    "Country": "Colombia",
    "SBF_Pa_s_Lead_Scoring__c": "Colombia",
    "State": "[Object: null]",
    "PostalCode": "[Object: null]",
    "Status": "SQL",
    "SBF_Fecha_Status__c": "[Object: null]",
    "CreatedDate": "2023-07-12T09:41:18.000+0000",
    "TimeToLive": 1690059799,
    "wb_date": "12/07/2023",
    "TimeQuery": "2023-07-13T10:55:32Z"
  },
  {
    "salesforce_id": "00Q7V00001tCDJGUAbncdncmhvyth",
    "type": "Lead",
    "event_name": "New Lead",
    "Email": "chinxxxxx@gmail.com",
    "Phone": "[Object: null]",
    "MobilePhone": "+519xxxxx3116",
    "Name": "Cinthya Zeas xxxxx",
    "FirstName": "Cinthya",
    "LastName": "Zeas torres",
    "City": "[Object: null]",
    "SBF_Pais__c": "Perú",
    "Country": "Perú",
    "SBF_Pa_s_Lead_Scoring__c": "Perú",
    "State": "[Object: null]",
    "PostalCode": "[Object: null]",
    "Status": "SQL",
    "SBF_Fecha_Status__c": "[Object: null]",
    "CreatedDate": "2023-07-12T09:43:07.000+0000",
    "TimeToLive": 1690059799,
    "wb_date": "12/07/2023",
    "TimeQuery": "2023-07-13T10:55:32Z"
  },
  {
    "salesforce_id": "",
    "type": "",
    "event_name": "",
    "Email": "",
    "Phone": "",
    "MobilePhone": "",
    "Name": "",
    "FirstName": "",
    "LastName": "",
    "City": "",
    "SBF_Pais__c": "",
    "Country": "",
    "SBF_Pa_s_Lead_Scoring__c": "",
    "State": "",
    "PostalCode": "",
    "Status": "",
    "SBF_Fecha_Status__c": "",
    "CreatedDate": "",
    "TimeToLive": 16852255,
    "wb_date": "",
    "TimeQuery": ""
  },
  {
    "salesforce_id": "",
    "type": "",
    "event_name": "",
    "Email": "",
    "Phone": "",
    "MobilePhone": "",
    "Name": "",
    "FirstName": "",
    "LastName": "",
    "City": "",
    "SBF_Pais__c": "",
    "Country": "",
    "SBF_Pa_s_Lead_Scoring__c": "",
    "State": "",
    "PostalCode": "",
    "Status": "",
    "SBF_Fecha_Status__c": "",
    "CreatedDate": "",
    "TimeToLive": 16852255,
    "wb_date": "",
    "TimeQuery": ""
  }
]
```
As you can see, my automation runs out of records on the last batch. So, the JSON ends up with empty fields.

The problem is that  when the request has empty values, it is rejected and none of the values are being written on the dynamo DB.

Is there a way to make the API gateway or Dynamo DB to ignore this empty values?

Answer

One possible solution is to place the API Gateway behind CloudFront.   With CloudFront in place, you can utilize Lambda@Edge functions which allow you to manipulate the request body using Lambda functions.  In this Lambda function, you could "clean up" the request body by removing the records with empty strings.

This link has general information on Lambda@Edge functions.
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-examples.html

Ideally, the JSON request would not contain the empty values.

BatchWritteItem in DynamoDB Ignore Empty Values

Relevant content