add column name as bucket folder in aws dms full-load task | source MySQL | target S3

Hello,

AWS DMS doesn't directly support dynamic folder creation based on column values.

Try To do these:

• Create a staging table in MySQL with a 'days' column.
• Configure DMS to extract from this table.
• Use a custom script to read data from S3, create folders based on 'days', and move data accordingly.
• Key points: Consider performance, error handling, and S3 bucket permissions.

EX: Create a Staging Table in MySQL

CREATE TABLE staging_table (
  column1 INT,
  column2 VARCHAR(255),
  -- other columns
  days INT
);

Populate the Staging Table

INSERT INTO staging_table (column1, column2, ...)
SELECT column1, column2, ...
FROM your_source_table;

Create an AWS DMS Task

• Configure the DMS task to extract data from the staging_table.
• Set the target S3 bucket and a generic folder structure (e.g., data/).