data quality on Dynamodb and Timestream
0
Hi everyone. what do you do for checking data quality on Dynamodb and Timestream? since in Timestream only dimensions need to be present,and in Dynamodb only Primary Key need to be present we might face with some data quality issue I would be grateful if give me your opinion
已提问 1 个月前52 查看次数
1 回答
- 最新
- 投票最多
- 评论最多
0
DynamoDB does not provide an out of the box solution for schema enforcement or validation. You must enforce your schema and data on the client side. You can do that through higher level SDKs such as DynamoDB Mapper or Enhanced Client, or through third party SDK Clients.
If you need something like DynamoDB but with schema enforcement then you can consider Amazon Keyspaces.
相关内容
AWS 官方已更新 3 个月前- AWS 官方已更新 4 个月前
- AWS 官方已更新 3 个月前
- AWS 官方已更新 1 年前
thanks. if i wanna checking data quality for Timestream and DDB with deequ or great expectations, can I export them and then use those tools? or can I use them directly?
I can't speak for Timestream, but DynamoDB allows you to export the table to S3 where you can run anything over the data and have no production impact on your table.
have you heard about using those data quality tools for ddb without exporting data? i mean connect to table directly
deequ is built into Glue, and Glue has connectors for DynamoDB so I assume its entirely possible: https://aws.amazon.com/blogs/big-data/test-data-quality-at-scale-with-deequ/