DynamoDB querying GSI, page.lastEvaluatedKey returns non-null/ non-empty key when there are no more items to be retrieved.

0

I have Project Table with a GSI on the userID.

For a user say TestUser, I have 10 projects in my Dynamo Table/ the GSI FilterByUserID.

I'm using awssdk QueryEnhancedRequest for my query to get these projects:

QueryEnhancedRequest.Builder requestBuilder = QueryEnhancedRequest.builder(); requestBuilder .exclusiveStartKey(<startKey> ) .queryConditional(<Query condition matching the PK of the GSI>) .limit(10) .build();

I then subscribe the pages using

SdkPublisher<Page<Project>> publisher
subscribe(page -> {})

page.lastEvaluatedKey() in this case is expected to be null as the first page returns all the available 10 items/ projects we had in the DDB. But surprisingly page.lastEvaluatedKey() return the 10th item from the GSI that matched the key.

I'm wondering why this would happen, as per documentation https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.Pagination.html

A single Query only returns a result set that fits within the 1 MB size limit. To determine whether there are more results, and to retrieve them one page at a time, applications should do the following: Examine the low-level Query result:

  1. If the result contains a LastEvaluatedKey element and it's non-null, proceed to step 2.
  2. If there is not a LastEvaluatedKey in the result, there are no more items to be retrieved.

Any help to understand this scenario would be appreciated. Thanks

질문됨 8달 전481회 조회
1개 답변
1

This is because you didn't allow DynamoDB to evaluate that there was more than 10. You had exactly 10, you asked for 10, but DynamoDB did not look for an 11th, so it returned you a LastEvaluatedKey which would have returned no items, as only when you return for the second page that DynamoDB continues to evaluate if there is 11,12..... items.

profile pictureAWS
전문가
답변함 8달 전
  • Thanks Leeroy Makes sense, but shouldn't the page.lastEvaluatedKey be independent of the limit over the query.

    If there is any documentation over this, that you might have handy please feel free to share with me.

  • You use .limit(10) which informs DynamoDB to obtain only 10 items. So the read request has no idea if more than 10 existed, so it provides you a pointer to the last read item, which is item 10. When it goes back, theres no more items and it now knows that, so LastEvaluatedKey is now empty/null.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠