Databrew Data Quality Queries

Question

Planning to use databrew for data validation. 
I not going to clean the data using Recipe, but just profile and write rule sets and then create a profile job to run on any new datsets. 
Could you please help me with the below:
1) can databrew work on streaming data?
2) can databrew profile job be run concurrently (at same time)  on 2 split files(both have same structure)

These are key to determine whether I can use databrew or not. Please advise.
Thank you!!

Answer

Hi,

1. Data brew does not support streaming data as of today.
2. You can run a profile job manually for each dataset, not at the same time.

Databrew Data Quality Queries

Relevant questions

Text data cleaning in databrew

Can you export Glue DataBrew recipies as code?

data brew capacity

Data Quality Framework in AWS

Trigger databrew job multiple times