DataQuality rules cannot be parsed

Question

Hi, i'm creating a resulset for a table mapped from SnowFlake, i want to apply a dataquality rule to that table, i'm not sure if it's possilbe.

Name                                                                  Database                                        Location                                          Classification
snowflake_sample_data_tpch_sf1_nation      testdb     SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.NATION     snowflake

when i try to save the rilset appears the next message:

Error encountered
InvalidInputException (status: 400): DataQuality rules cannot be parsed
See details
{ "message": "DataQuality rules cannot be parsed", "code": "InvalidInputException", "time": "2023-05-21T21:28:57.097Z", "requestId": "XXX", "statusCode": 400, "retryable": false }

Cheers

Answer

Hello,
Warm greetings from AWS!

This error generally occurs if the required permissions are not granted to the user setting the DQ rules or there is some issue with DQ rules arguments.

I would suggested you to try the following Glue AWS CLI command to create the DQ rule:

For e.g.
aws glue create-data-quality-ruleset --name yellow-stone-dq --ruleset 'Rules= [IsComplete "event_key"]' --target-table '{"TableName":"noun-facts-json-table", "DatabaseName":"prod-yellow-stone"}'

Ref.
[+]  https://docs.aws.amazon.com/cli/latest/reference/glue/create-data-quality-ruleset.html

To know about the DQ DL syntax and Rule structure, please refer to this doc [+] https://docs.aws.amazon.com/glue/latest/dg/dqdl.html

In addition to above rules, IAM permissions required to add the rules are described in this doc [+] https://docs.aws.amazon.com/glue/latest/dg/data-quality-authorization.html

other documentations that you can find helpful are as follows:

Tutorial of how to get started with DQ - https://docs.aws.amazon.com/glue/latest/ug/tutorial-data-quality.html    
 Blog in Depth breakdown of DQ along with guide  - https://aws.amazon.com/blogs/aws/join-the-preview-aws-glue-data-quality/

DataQuality rules cannot be parsed

相關內容