AWD DMS setting CDC file size

0

Hi Everyone,

I configured an AWS DMS to migrate data from an AWS Managed Oracle database to a S3 Bucket.

The CDC has been creating file greater than 32MB. I was expecting to have file with at most 32MB due to the cdcMinFileSize. In my bucket there are files greater than 200MB.

cdcMaxBatchInterval - The maximum interval length condition, defined in seconds, to output a file to Amazon S3. The default value is 60 seconds.

cdcMinFileSize - The minimum file size condition, defined in KB, to output a file to Amazon S3. The default value is 32000 KB.

WriteBufferSize - The size, in KB, of the in-memory file write buffer used when generating .csv files on the local disk at the AWS DMS replication instance. The default value is 1000 KB.

I was wondering if my oracle database committed a large transaction that lead to this scenario. But during the processing on the Replication Instance would break the file into chunks to fit the 32MB?

Thanks in advance.

Marcus
preguntada hace 2 años667 visualizaciones
1 Respuesta
0

cdcMinFileSize defines the least or smallest file that AWS DMS will create on S3. If you are seeing a 200 MB file, its expected. Bottom line, with the default setting of 32000 (KB) for cdcMinFileSize, you will not see any files on S3 smaller than 32000 KB.

AWS
D-Rao
respondido hace 2 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas