Scan Dynamo DB using ExclusiveStartKey to return the 1st record that matches the filter

1

Hello there. I'm having this challenge where my Scan request mapping looks like this:

{
    "TableName": "Test-Table",
    "Limit": 10000,
    "FilterExpression": "wb_date = :val",
    "ExpressionAttributeValues": {
        ":val": {
            "S": "$input.params('wb_date')"
        }
    },
    "ExclusiveStartKey": {
        "salesforce_id": {
            "S": "$input.params('salesforce_id')"
        },
        "type": {
            "S": "$input.params('type')"
        }
    },
    "ReturnConsumedCapacity": "TOTAL"
}

My request and response are retrieving the data. However, I need to know how to make that first API Call to retrieve the first matching record. If I pass 0 on my request https://i.imgur.com/qDZeBpf.png, the data returned starts from any other record than the first value of the DynamoDB.

Since this is my 1st API call, I do not know what values to add in the ExclusiveStartKey params (salesforce_id and type). So my question aims to know if there is some sort of wildcard or default value to retrieve the data from the default 1st record found. Then, I can continue to retrieve based on the LastEvaluatedKey response.

Thanks!

2 Answers
1

The "ExclusiveStartKey" setting is optional.
So, it is not necessary to specify it at the very first execution.
LastEvaluatedKey is included in the response when all data could not be retrieved during the first scan.
The first run will scan without specifying ExclusiveStartKey.
If the response from the first run contains LastEvaluatedKey, put a value in ExclusiveStartKey and run it again.
So I think you need to change the content of your API a bit.
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html

profile picture
EXPERT
answered 9 months ago
  • Thanks a lot! It worked. I just have to modify mi API request mapping by removing the ExclusiveStartKey

  • Hey Riku. After testing it, if ExclusiveStartKey is not declared in the Mapping Request, the params do not pass to the API. So that, the scan does not start where it should.

    I deleted the ExclusiveStartKey field from my JSON:

    { "TableName": "Test-Table", "Limit": 10000, "FilterExpression": "wb_date = :val", "ExpressionAttributeValues": { ":val": { "S": "$input.params('wb_date')" } }, "ReturnConsumedCapacity": "TOTAL" }

    Maybe I understood wrong. Is this what you meant?

  • Yes, it is. The following document states that the value used for "ExclusiveStartKey" uses "LastEvaluatedKey". In other words, it is a parameter for retrieving the data that could not be retrieved in the first request from the continuation. https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html#DDB-Scan-request-ExclusiveStartKey

    The primary key of the first item that this operation will evaluate. Use the value that was returned for LastEvaluatedKey in the previous operation.

1

Since the ExclusiveStartKey attribute is optional. I just had to modify mi API request mapping by removing the ExclusiveStartKey like this:

{
    "TableName": "Test-Table",
    "Limit": 10000,
    "FilterExpression": "wb_date = :val",
    "ExpressionAttributeValues": {
        ":val": {
            "S": "$input.params('wb_date')"
        }
    },
      "ReturnConsumedCapacity": "TOTAL"
}

Despite removing it, the API still process it when I consume it with 0 value or with a real Id value

Lucas
answered 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions