Dynamodb : Need sorted + filtered + paginated data

0

Hi All,

Need some help around dynamodb.

We have an app which process the a batch of 44K records and saves the data to DDB table. This has

  1. PK (run_id_random_number)
  2. SK (run_id)

We want to fetch the filtred data page by page and in a sorted way .

Lets say there 20 attibutes pre row and we have to filter the data by region,country,status for example region='Europe' & country='Luxembourg' the total number of records come down to 20K, and have to sort on status attiribte.

How can I get records in a paginated manner(page size is 500) ?

Thanks, Ashish

Ashish
질문됨 일 년 전1111회 조회
1개 답변
0
수락된 답변

You must use Scan with a FilterExpression to achieve your use case.

You can set the Limit to 500, but keep in mind that the Filter is applied after the limit is reached, so you are only guaranteed ~500 items, not exactly 500.

A single Scan operation reads up to the maximum number of items set (if using the Limit parameter) or a maximum of 1 MB of data and then apply any filtering to the results using FilterExpression. If LastEvaluatedKey is present in the response, you need to paginate the result set. For more information, see Paginating the Results in the Amazon DynamoDB Developer Guide.

this will filter the data , I need the data to be ordred by one of the columns In this case , I need the data to be sorted by 'status' attribute ,, status attribute can have values (PASS,FAILED,EVALUATING,DEGRADING,<>)

If you need all your data ordered by status, then you would have to create a GSI with a static partition key and status as sort key.

GSI_PKGSI_SKData
1DEGRADINGData
1PASSData
1PASSData

This would ensure you can get the data returned sorted by the sort key as DynamoDB does not provide sorting across partition keys. Keep in mind this solution would limit your writes to 1000 WCU per second. If you require more you can shard the partition key.

profile pictureAWS
전문가
답변함 일 년 전
  • this will filter the data , I need the data to be ordred by one of the columns In this case , I need the data to be sorted by 'status' attribute ,, status attribute can have values (PASS,FAILED,EVALUATING,DEGRADING,<>)

  • Updated my answer.

  • Thank you , it there a possibility to sort by multiple attibutes

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠