Record Counts in Parquet and SQL

0

Should the parquet files created by DMS contain the same number of records as the SQL change table? Very helpful to know in order to compare records in SQL log to that in the S3 parquet. .

AWS
已提问 10 个月前271 查看次数
1 回答
0

Hi, If you look at section CdcInsertsOnly of https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html, it clearly says that UPDATEs, INSERTs and DELETEs are reported in parquet files with one row per change.

By default (the false setting), the first field in a .csv or .parquet record contains
 the letter I (INSERT), U (UPDATE), or D (DELETE). These values indicate whether 
the row was inserted, updated, or deleted at the source database for a CDC load 
to the target.

So, all changes are clearly reported.

I personally use not with Parquet but CSV files and it works fine.

Best, Didier

profile pictureAWS
专家
已回答 10 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则

相关内容