NextToken from ExecuteStatement returns error "NextToken does not match request" in Lambda function

1

Running into issue with ExecuteStatement. The result returned both "LastEvaluatedKey" and "NextToken" but when we fed it into the next "ExecuteStatement" as part of "input", we keep getting an error "NextToken does not match request".

const input = { Statement: '...', Parameter: [...] };
let end = false;
let nextInput = input;
while (!end) {
const { LastEvaluatedKey, NextToken, ...rest } =
  await new ExecuteStatementCommand(nextInput);
LastEvaluatedKey === undefined ?
  end = true :
  nextInput = { NextToken, ...input };
}

Saw this other question regarding LastEvaluatedKey which was helpful. However, the proposed solution does not work as our key value is date based (i.e. pulling data based on a date range). So, the query result ended with numerous duplicate items created on same date.

  • Don't know why the NextToken is not working for you. Keep in mind that DynamoDB does not store a cursor it can continue with. NextToken is actually decryptable by DynamoDB to get a LastEvaluatedKey. DynamoDB is stateless.

    If LastEvaluatedKey is not unique tablewide for the index or the table key it is a problem in DynamoDB. You might want to reconsider the tables design. What I ended up doing in such a case if appending a random GUID to the date field to make it unique. You can only do this if the date is in a range key though, otherwise you can never select 1 record :-)

    Strangly this ExecuteStatement does not accept an ExclusiveKeyStart... I would try passing both the NextToken and the ExclusiveKeyStart=LastEvaluatedKey.

2 Answers
0

To JaccoPK, ExclusiveKeyStart is not an input for ExecuteStatementCommand unfortunately.

answered 2 years ago
0

I found this error on my application, after some debugging I found it was due to the query statement being built in a different order even though the filters are the same, example:

SELECT * FROM "SomeTable"."valueOneGSI" where "valueOne" = 'someValue' AND "valueTwo" IS MISSING AND "valueThree" = 'someOtherValue'

vs

SELECT * FROM "SomeTable"."valueOneGSI" where "valueOne" = 'someValue' AND "valueThree" = 'someOtherValue' AND "valueTwo" IS MISSING

In my case it happened because I was receiving the filters in the body but the order was changed due to NextToken field being included in subsequent requests, so I just made sure to always add the AND clauses in alphabetical order and it's working properly now.

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions