Controlling Frequency of Writes to S3 Target With DMS

Hello Weves,

Thanks for reaching out and it seems you are trying to control CDC with S3 target. Please note that batch apply mode is not applicable or supported with S3 target.

You can use below internal parameters ( as target endpoint extra connection attributes) to control the way changes are applied on target during replication.

cdcMaxBatchInterval default value 60 seconds
cdcMinFileSize default value 32 MB

Please change values for above parameters and it will help you change how csv file is generated during CDC.

I hope this information helps... I will try to get this parameter documented on our public docs..

Thanks,
Mahesh Kansara

Edited by: MaheshKansara on May 4, 2020 4:58 PM

Thanks. The cdcMaxBatchInterval param worked pretty well for us.

What are the measurement units for these params? And are they documented anywhere in AWS docs yet?? Can't find them.
cdcMaxBatchInterval - Is this in seconds?
cdcMinFileSize - Is this in MB? KB?

Hi pcguy166,

You're correct, cdcMaxBatchInterval is in seconds, and cdcMinFileSize is in MB. This is documented in the "Extra connection attributes when using Amazon S3 as a target for AWS DMS" section of the User Guide, which you can access via the link below.

https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.S3.html#CHAP_Target.S3.Configuring

Best,
Sean