How do I add or modify endpoint settings for AWS DMS endpoints?

Resolution

Use one of the following methods to add or modify the endpoint settings for your AWS DMS endpoints:

• AWS DMS console
• AWS CloudFormation
• AWS Command Line Interface (AWS CLI)

Note: If you receive errors when you run AWS CLI commands, then see Troubleshoot AWS CLI errors. Also, make sure that you're using the most recent AWS CLI version.

Use the AWS DMS console to create a new endpoint with extra connection attributes

1. Open the AWS DMS console.

2. Select the AWS Region where you want to create the endpoint.

3. In the navigation pane, choose Endpoints.

4. Choose Create endpoint.

5. Choose the Endpoint type, and then enter the Endpoint configuration details.

6. Expand the Endpoint settings section.

7. Choose Add new setting. Use the wizard or JSON editor to provide the name and value for the endpoint.
   Example:

   {
     "CsvRowDelimiter": "\\n",
     "CsvDelimiter": ",",
     "CompressionType": "GZIP",
     "DataFormat": "parquet",
     "EnableStatistics": true,
     "DatePartitionEnabled": false
   }

8. (Optional) Choose Use endpoint connection attributes. Enter your extra connection attributes (ECAs). Specify each attribute as a name-value pair associated by an equal sign (=). Separate multiple attributes with a semicolon (;) with no additional white space.
   Example:

   compressionType=GZIP;csvDelimiter=,;csvRowDelimiter=\n;datePartitionEnabled=false;
   

9. Expand the Test endpoint connection section.

10. Choose your Replication instance.

11. Choose Run test.

12. Choose Create endpoint.

Use the AWS DMS console to add endpoint settings or ECAs to an existing endpoint

1. Open the AWS DMS console.
2. Select the Region that the endpoint is in.
3. From the navigation pane, choose Endpoints, and then select the endpoint that you want to modify.
4. Choose Actions, and then choose Modify.
5. Expand the Endpoint settings section, and then use the wizard or JSON editor to enter the endpoint setting Key-Values.
6. Choose Save.
7. Select the endpoint again.
8. Choose Connection tab, and then choose Test connections.
9. Select your Replication instance, and then choose Run test.

Use the AWS CLI to create a new endpoint with custom endpoint settings

1. Install and configure the AWS CLI with the Region where you want to create the endpoint. For example, set your Region with the --region parameter.

2. Run the following create-endpoint command:

   aws dms create-endpoint --endpoint-identifier VALUE --endpoint-type VALUE --engine-name VALUE --s3-settings ServiceAccessRoleArn=arn:aws:iam::1111111111111:role/ROLE-NAME,BucketName=BUCKET-NAME,CompressionType=GZIP,MaxFileSize=32000

3. Run the following test-connection command to test the connectivity of the new endpoint with the replication instance:

   aws dms test-connection \
       --replication-instance-arn arn:aws:dms:us-east-1:123456789012:rep:EXAMPLE-REPLICATION-INSTANCE \
       --endpoint-arn arn:aws:dms:us-east-1:123456789012:endpoint:EXAMPLE-ENDPOINT

   Note: In the preceding commands, replace VALUE, ROLE-NAME, and BUCKET-NAME with your values. Replace EXAMPLE-REPLICATION-INSTANCE with the Amazon Resource Name (ARN) of your replication instance and EXAMPLE-ENDPOINT with the ARN of your endpoint.

Use the AWS CLI to add endpoint settings to an existing endpoint

1. Install and configure the AWS CLI with the Region where you want to modify the endpoint. For example, set your Region with the --region parameter.

2. Run the following modify-endpoint command:

   aws dms modify-endpoint --endpoint-arn VALUE --s3-settings Rfc4180=true,AddColumnName=true,CdcInsertsOnly=true,MaxFileSize=64000
   

3. Run the following test-connection command to test the connectivity of the modified endpoint with the replication instance:

   aws dms test-connection \
       --replication-instance-arn arn:aws:dms:us-east-1:123456789012:rep:EXAMPLE-REPLICATION-INSTANCE \
       --endpoint-arn arn:aws:dms:us-east-1:123456789012:endpoint:EXAMPLE-ENDPOINT

   Note: Replace EXAMPLE-REPLICATION-INSTANCE with the ARN of your replication instance and EXAMPLE-ENDPOINT with the ARN of your endpoint.

Use CloudFormation to create a new endpoint with custom endpoint settings

1. Open the CloudFormation console.

2. Select the Region where you want to create the endpoint.

3. Choose Create stack, and then choose With new resources (standard).

4. Select Choose an existing template.

5. Under Specify template, choose Upload a template file. Use the following example template:

   AWSTemplateFormatVersion: 2010-09-09
   Description: "S3 target Endpoint"
   Resources:
     BasicS3TargetEndpoint:
       Type: AWS::DMS::Endpoint
       Properties:
         EndpointIdentifier: s3-Target-endpoint
         EndpointType: target
         EngineName: s3
         S3Settings:
           ServiceAccessRoleArn: arn:aws:iam::111122223333:role/ROLE-NAME
           BucketName: BUCKET-NAME
           AddColumnName: true
           CdcInsertsOnly: true
           MaxFileSize: 32000

   Note: Replace 111122223333 with your AWS account ID and ROLE-NAME with your IAM role name that has access to the Amazon Simple Storage Service (Amazon S3) bucket.

6. Choose Next.

7. Enter a name for the stack.

8. Assign an AWS Identity and Access Management (IAM) role with the required permissions to create DMS resources.

9. Choose Next.

10. Choose Submit.

Use CloudFormation to add endpoint settings to an existing endpoint

1. Open the CloudFormation console.

2. Select the Region where you want to create the endpoint.

3. Select an existing stack.

4. Choose Update stack, and then choose Make a direct update.

5. On the Update stack page, choose Replace existing template.

6. Under Specify template, choose Upload a template file. Use the following example template:

   AWSTemplateFormatVersion: 2010-09-09
   Description: "S3 target Endpoint"
   Resources:
     S3TargetEndpoint:
       Type: AWS::DMS::Endpoint
       Properties:
         EndpointIdentifier: s3-Target-endpoint
         EndpointType: target
         EngineName: s3
         S3Settings:
           ServiceAccessRoleArn: arn:aws:iam::111122223333:role/ROLE-NAME
           BucketName: BUCKET-NAME
           AddColumnName: true
           CdcInsertsAndUpdates: true
           MaxFileSize: 64000

   Note: Replace 111122223333 with your account ID and ROLE-NAME with your IAM role name that has access to the Amazon S3 bucket.

7. Choose Next.

8. On the Specify stack details page, choose Next.

9. (Optional) On the Configure stack options page, under Tags, add tags. Under Permissions, select the IAM role that you want to make updates on your behalf. Then, choose Next.

10. Choose Submit.

Use legacy ECAs to create or modify an endpoint

You can use the --extra-connection-attributes option to create and modify the endpoint. Specify each attribute as a name-value pair associated by an equal sign (=). Separate multiple attributes with a semicolon (;) with no additional white space.

Example:

aws dms create-endpoint \
    --endpoint-identifier YOUR-VALUE \
    --endpoint-type target \
    --engine-name s3 \
    --service-access-role-arn arn:aws:iam::YOUR-ACCOUNT-ID:role/YOUR-ROLE-NAME \
    --extra-connection-attributes "bucketName=YOUR-BUCKET-NAME;encryptionMode=SSE_KMS;maxFileSize=512"

aws dms modify-endpoint \
    --endpoint-arn arn:aws:dms:us-east-1:123456789012:endpoint:EXAMPLE \
    --extra-connection-attributes "encryptionMode=SSE_KMS;maxFileSize=512"

Related information

Working with AWS DMS endpoints

AWS::DMS::Endpoint