- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
For the custom SQL rule question, when AWS Glue Data Quality evaluates the custom SQL rule:
It will run the SQL query on the dataframe. In this case "select Name from primary where Age > 18".
It will return the rows that satisfy the condition i.e Age > 18.
If there are multiple rows for the same Name that satisfy the condition, it will return both rows. For example, if there are two rows for name "Manish", one with age 16 and other with age 23, it will return the row with age 23 since that satisfies the condition but not the row with age 16.
So in this case, it will pass the row with name "Manish" and age 23 but fail the row with the same name "Manish" but age 16.
The documentation does not explicitly mention the behavior for multiple rows with same value, but based on how custom SQL rules work, it is expected to behave in this way.
Contenuto pertinente
- AWS UFFICIALEAggiornata 9 mesi fa
- AWS UFFICIALEAggiornata 3 anni fa
No as per my test cases this is not working as you described. Glue DQ is passing both the rows with name Manish. If I change one name to Rajesh and keep one Manish, it works as expected.