Skip to content

redshift serverless endpoint

0

I want to configure 10 dms tasks with single common redshift serverless endpoint source endpoint will be different but target will be same want to do by cdk how is it possible to use single common redshift endpoint for all 10 tasks as we need to pass s3 bucket in redshift we cannot pass diff task s3 buckets in single target endpoint, also if possible can we use single s3 bucket also for all 10 tasks like single common target endpoint

asked 10 months ago170 views

1 Answer
0

Yes, you can configure 10 DMS tasks with a single common Redshift Serverless endpoint as the target. This is entirely possible because AWS DMS allows you to reuse endpoints across multiple tasks. When you have multiple tasks that use the same target endpoint, that endpoint only counts once toward the 100-endpoint limit per replication instance.

For your Redshift Serverless target endpoint configuration in CDK, you'll need to:

  1. Create a single Redshift Serverless target endpoint with the correct configuration:
const redshiftServerlessTarget = new dms.CfnEndpoint(this, 'RedshiftServerlessTarget', {
endpointType: 'target',
engineName: 'redshift-serverless',
endpointIdentifier: 'redshift-serverless-target',
serverName: redshiftServerlessWorkgroup.endpoint.address,
port: 5439,
databaseName: 'your_database',
username: 'admin_username',
password: 'admin_password',
redshiftSettings: {
serverName: redshiftServerlessWorkgroup.endpoint.address
}
});
  1. Create your 10 different source endpoints as needed.

  2. Create 10 DMS tasks, each referencing its unique source endpoint but all pointing to the same Redshift Serverless target endpoint:

for (let i = 0; i < 10; i++) {
new dms.CfnReplicationTask(this, `ReplicationTask${i}`, {
migrationType: 'full-load-and-cdc',
replicationInstanceArn: replicationInstance.attrReplicationInstanceArn,
sourceEndpointArn: sourceEndpoints[i].attrEndpointArn,
targetEndpointArn: redshiftServerlessTarget.attrEndpointArn,
tableMappings: JSON.stringify({
// Your table mappings
}),
// Other task settings
});
}

Regarding S3 buckets: Yes, you can use a single S3 bucket for all 10 tasks. DMS tasks can share the same S3 bucket for intermediate storage. You can organize the data within the bucket using different folder paths for each task to avoid conflicts. This approach simplifies management and reduces the number of resources you need to create and maintain.

When configuring your tasks, you can specify different folder paths within the same S3 bucket for each task to keep the data organized and separate. This way, you maintain a clean separation of data while still using a single common S3 bucket and a single Redshift Serverless target endpoint.
Sources
Create DMS target endpoint via Terraform - Redshift Serverless | AWS re:Post
Understanding DMS replication instance endpoint limit | AWS re:Post

answered 10 months ago

AWS
SUPPORT ENGINEER

reviewed 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.