data quality on Dynamodb and Timestream
0
Hi everyone. what do you do for checking data quality on Dynamodb and Timestream? since in Timestream only dimensions need to be present,and in Dynamodb only Primary Key need to be present we might face with some data quality issue I would be grateful if give me your opinion
已提問 1 個月前檢視次數 56 次
1 個回答
- 最新
- 最多得票
- 最多評論
0
DynamoDB does not provide an out of the box solution for schema enforcement or validation. You must enforce your schema and data on the client side. You can do that through higher level SDKs such as DynamoDB Mapper or Enhanced Client, or through third party SDK Clients.
If you need something like DynamoDB but with schema enforcement then you can consider Amazon Keyspaces.
相關內容
AWS 官方已更新 2 年前- AWS 官方已更新 1 年前
- AWS 官方已更新 4 個月前
thanks. if i wanna checking data quality for Timestream and DDB with deequ or great expectations, can I export them and then use those tools? or can I use them directly?
I can't speak for Timestream, but DynamoDB allows you to export the table to S3 where you can run anything over the data and have no production impact on your table.
have you heard about using those data quality tools for ddb without exporting data? i mean connect to table directly
deequ is built into Glue, and Glue has connectors for DynamoDB so I assume its entirely possible: https://aws.amazon.com/blogs/big-data/test-data-quality-at-scale-with-deequ/