Dynamodb : Need sorted + filtered + paginated data

0

Hi All,

Need some help around dynamodb.

We have an app which process the a batch of 44K records and saves the data to DDB table. This has

  1. PK (run_id_random_number)
  2. SK (run_id)

We want to fetch the filtred data page by page and in a sorted way .

Lets say there 20 attibutes pre row and we have to filter the data by region,country,status for example region='Europe' & country='Luxembourg' the total number of records come down to 20K, and have to sort on status attiribte.

How can I get records in a paginated manner(page size is 500) ?

Thanks, Ashish

Ashish
質問済み 1年前1111ビュー
1回答
0
承認された回答

You must use Scan with a FilterExpression to achieve your use case.

You can set the Limit to 500, but keep in mind that the Filter is applied after the limit is reached, so you are only guaranteed ~500 items, not exactly 500.

A single Scan operation reads up to the maximum number of items set (if using the Limit parameter) or a maximum of 1 MB of data and then apply any filtering to the results using FilterExpression. If LastEvaluatedKey is present in the response, you need to paginate the result set. For more information, see Paginating the Results in the Amazon DynamoDB Developer Guide.

this will filter the data , I need the data to be ordred by one of the columns In this case , I need the data to be sorted by 'status' attribute ,, status attribute can have values (PASS,FAILED,EVALUATING,DEGRADING,<>)

If you need all your data ordered by status, then you would have to create a GSI with a static partition key and status as sort key.

GSI_PKGSI_SKData
1DEGRADINGData
1PASSData
1PASSData

This would ensure you can get the data returned sorted by the sort key as DynamoDB does not provide sorting across partition keys. Keep in mind this solution would limit your writes to 1000 WCU per second. If you require more you can shard the partition key.

profile pictureAWS
エキスパート
回答済み 1年前
  • this will filter the data , I need the data to be ordred by one of the columns In this case , I need the data to be sorted by 'status' attribute ,, status attribute can have values (PASS,FAILED,EVALUATING,DEGRADING,<>)

  • Updated my answer.

  • Thank you , it there a possibility to sort by multiple attibutes

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ