Record Counts in Parquet and SQL

0

Should the parquet files created by DMS contain the same number of records as the SQL change table? Very helpful to know in order to compare records in SQL log to that in the S3 parquet. .

AWS
demandé il y a 10 mois271 vues
1 réponse
0

Hi, If you look at section CdcInsertsOnly of https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html, it clearly says that UPDATEs, INSERTs and DELETEs are reported in parquet files with one row per change.

By default (the false setting), the first field in a .csv or .parquet record contains
 the letter I (INSERT), U (UPDATE), or D (DELETE). These values indicate whether 
the row was inserted, updated, or deleted at the source database for a CDC load 
to the target.

So, all changes are clearly reported.

I personally use not with Parquet but CSV files and it works fine.

Best, Didier

profile pictureAWS
EXPERT
répondu il y a 10 mois

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions