Record Counts in Parquet and SQL
0
Should the parquet files created by DMS contain the same number of records as the SQL change table? Very helpful to know in order to compare records in SQL log to that in the S3 parquet. .
질문됨 10달 전271회 조회
1개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
0
Hi, If you look at section CdcInsertsOnly of https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html, it clearly says that UPDATEs, INSERTs and DELETEs are reported in parquet files with one row per change.
By default (the false setting), the first field in a .csv or .parquet record contains
the letter I (INSERT), U (UPDATE), or D (DELETE). These values indicate whether
the row was inserted, updated, or deleted at the source database for a CDC load
to the target.
So, all changes are clearly reported.
I personally use not with Parquet but CSV files and it works fine.
Best, Didier
