Planning to use databrew for data validation. I not going to clean the data using Recipe, but just profile and write rule sets and then create a profile job to run on any new datsets. Could you please help me with the below: 1) can databrew work on streaming data? 2) can databrew profile job be run concurrently (at same time) on 2 split files(both have same structure) These are key to determine whether I can use databrew or not. Please advise. Thank you!!

Hi, 1. Data brew does not support streaming data as of today. 2. You can run a profile job manually for each dataset, not at the same time.

Databrew Data Quality Queries

Planning to use databrew for data validation. I not going to clean the data using Recipe, but just profile and write rule sets and then create a profile job to run on any new datsets. Could you please help me with the below:

can databrew work on streaming data?
can databrew profile job be run concurrently (at same time) on 2 split files(both have same structure)

These are key to determine whether I can use databrew or not. Please advise. Thank you!!

主题

分析

标签

AWS Glue DataBrew

语言

English

AWS-User-6839064

已提问 2 年前280 查看次数

1 回答

最新
投票最多
评论最多

这些答案有用吗？为正确答案投票，以帮助社区从您的知识中受益。

Hi,

Data brew does not support streaming data as of today.
You can run a profile job manually for each dataset, not at the same time.

CHELSEA IP

已回答 2 年前

专家

Julia Beck

已审核 2 年前

Databrew Data Quality Queries

相关内容