By using AWS re:Post, you agree to the Terms of Use

Tracking record-level failure from Databrew validation

0

Hi,

This is question with Databrew data profile capability. I am looking for record-level tracking per rule validation status. This will provide while rules has failed on a record from the dataset.

At least, I am looking to separate records after data profiling, valid vs invalid records.

Thanks,

asked 4 months ago26 views
1 Answers
0

Hello, this functionality is currently not supported, I will provide feedback to the team owning the service.

Today, to solve your issue you could use a implement in a project recipe the same logic in the quality check you are interested in and using a conditional transform associate a value (true/False or 1/0) to a new column isCol1Valid .

when you run the processing job and write the output to your target you could easily find all rows that are not satisfying the specific check.

I am aware this is currently a workaround.

hope it helps

EXPERT
answered 4 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions