Dynamodb : Need sorted + filtered + paginated data

0

Hi All,

Need some help around dynamodb.

We have an app which process the a batch of 44K records and saves the data to DDB table. This has

  1. PK (run_id_random_number)
  2. SK (run_id)

We want to fetch the filtred data page by page and in a sorted way .

Lets say there 20 attibutes pre row and we have to filter the data by region,country,status for example region='Europe' & country='Luxembourg' the total number of records come down to 20K, and have to sort on status attiribte.

How can I get records in a paginated manner(page size is 500) ?

Thanks, Ashish

Ashish
已提问 1 年前1112 查看次数
1 回答
0
已接受的回答

You must use Scan with a FilterExpression to achieve your use case.

You can set the Limit to 500, but keep in mind that the Filter is applied after the limit is reached, so you are only guaranteed ~500 items, not exactly 500.

A single Scan operation reads up to the maximum number of items set (if using the Limit parameter) or a maximum of 1 MB of data and then apply any filtering to the results using FilterExpression. If LastEvaluatedKey is present in the response, you need to paginate the result set. For more information, see Paginating the Results in the Amazon DynamoDB Developer Guide.

this will filter the data , I need the data to be ordred by one of the columns In this case , I need the data to be sorted by 'status' attribute ,, status attribute can have values (PASS,FAILED,EVALUATING,DEGRADING,<>)

If you need all your data ordered by status, then you would have to create a GSI with a static partition key and status as sort key.

GSI_PKGSI_SKData
1DEGRADINGData
1PASSData
1PASSData

This would ensure you can get the data returned sorted by the sort key as DynamoDB does not provide sorting across partition keys. Keep in mind this solution would limit your writes to 1000 WCU per second. If you require more you can shard the partition key.

profile pictureAWS
专家
已回答 1 年前
  • this will filter the data , I need the data to be ordred by one of the columns In this case , I need the data to be sorted by 'status' attribute ,, status attribute can have values (PASS,FAILED,EVALUATING,DEGRADING,<>)

  • Updated my answer.

  • Thank you , it there a possibility to sort by multiple attibutes

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则