- Newest
- Most votes
- Most comments
Yes, you can configure 10 DMS tasks with a single common Redshift Serverless endpoint as the target. This is entirely possible because AWS DMS allows you to reuse endpoints across multiple tasks. When you have multiple tasks that use the same target endpoint, that endpoint only counts once toward the 100-endpoint limit per replication instance.
For your Redshift Serverless target endpoint configuration in CDK, you'll need to:
- Create a single Redshift Serverless target endpoint with the correct configuration:
const redshiftServerlessTarget = new dms.CfnEndpoint(this, 'RedshiftServerlessTarget', { endpointType: 'target', engineName: 'redshift-serverless', endpointIdentifier: 'redshift-serverless-target', serverName: redshiftServerlessWorkgroup.endpoint.address, port: 5439, databaseName: 'your_database', username: 'admin_username', password: 'admin_password', redshiftSettings: { serverName: redshiftServerlessWorkgroup.endpoint.address } });
-
Create your 10 different source endpoints as needed.
-
Create 10 DMS tasks, each referencing its unique source endpoint but all pointing to the same Redshift Serverless target endpoint:
for (let i = 0; i < 10; i++) { new dms.CfnReplicationTask(this, `ReplicationTask${i}`, { migrationType: 'full-load-and-cdc', replicationInstanceArn: replicationInstance.attrReplicationInstanceArn, sourceEndpointArn: sourceEndpoints[i].attrEndpointArn, targetEndpointArn: redshiftServerlessTarget.attrEndpointArn, tableMappings: JSON.stringify({ // Your table mappings }), // Other task settings }); }
Regarding S3 buckets: Yes, you can use a single S3 bucket for all 10 tasks. DMS tasks can share the same S3 bucket for intermediate storage. You can organize the data within the bucket using different folder paths for each task to avoid conflicts. This approach simplifies management and reduces the number of resources you need to create and maintain.
When configuring your tasks, you can specify different folder paths within the same S3 bucket for each task to keep the data organized and separate. This way, you maintain a clean separation of data while still using a single common S3 bucket and a single Redshift Serverless target endpoint.
Sources
Create DMS target endpoint via Terraform - Redshift Serverless | AWS re:Post
Understanding DMS replication instance endpoint limit | AWS re:Post
Relevant content
asked 10 months ago
asked a year ago
