Dynamodb query with filters returns no items

0

I have this table:

id(string)message(string)created(number)completed(boolean)
123Instruction1686731293161-
............
129Instruction1686821275413false

Partition key: message

Sort key: created

Projected keys: All

In summary for the table:

  • some entries does not have message and completed attributes
  • even though message is used as partition key in GSI, this is ok for my use case

I tried to query using aws-sdk dynamodb client with this params:

  TableName: "test-idStatus",
  IndexName: "messageCreatedIndex",
  KeyConditionExpression: "#message = :message and #created BETWEEN :createdfrom AND :createdto",
  ExpressionAttributeNames: {
    "#message": "message",
    "#created": "created",
    "#completed": "completed",
  },
  ExpressionAttributeValues: {
    ":messageFunction": "Instruction",
    ":createdfrom": 1686817506000, // 2023-June-15 08:25
    ":createdto": 1686821106000,   // 2023-June-15 09:25
    ":completed": "false",
  },
  ScanIndexForward: false,
  Limit: 5,
  Select: "ALL_ATTRIBUTES",
  FilterExpression: "#completed = :completed",
}

This returned entries

But when I changed the ExpressionAttributeValues of:

createdfrom: 1686731286000 // (2023-June-14 08:25)

This did not return any entries (but returns a LastEvaluatedKey).

Which is weird since June-15 returned entries and is just a subset of June-14 to June-15 entries

I also tried to replicate this in the AWS console dynamoDB. The same thing happened.

Then I also tried removing ScanIndexForward, now it returned entries.

질문됨 일 년 전813회 조회
2개 답변
1

What's happening here is that you set a FilterExpression and a Limit. What happens now is that DynamoDB reads 5 items as that is your Limit and then applies your filter to those 5 items but none match the expression so you get an empty response.

Changing the date is simply changing the 5 items read and it's just luck that they match your filter. The same goes for changing the scan direction, you're just changing the first 5 items.

https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html

A single Query operation will read up to the maximum number of items set (if using the Limit parameter) or a maximum of 1 MB of data and then apply any filtering to the results using FilterExpression. If LastEvaluatedKey is present in the response, you will need to paginate the result set. For more information, see Paginating the Results in the Amazon DynamoDB Developer Guide.

profile pictureAWS
전문가
답변함 일 년 전
  • Why is it that it reads first 5 items and then filters and not the other way around? Do we have some workaround for this scenario?

  • This is how it works, as outlined in the documentation and edited my answer to highlight. You need to include pagination.

-1
답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠