How can I add or modify endpoint settings for AWS DMS endpoints?

5 minuti di lettura
1

I want to add or modify extra connection attributes for an AWS Database Migration Service (AWS DMS) endpoint.

Short description

Endpoint Settings (formerly Extra Connection Attributes) define additional endpoint-specific settings to use for an AWS DMS Task. For example, use endpoint settings to control the timeout, time zone, and file size settings of a specific endpoint. To add or modify the endpoint settings for your AWS DMS endpoint, use these options:

  • AWS DMS console
  • AWS Command Line Interface (AWS CLI)
  • AWS CloudFormation.

Resolution

Note: If you receive errors when you run AWS CLI commands, then see Troubleshoot AWS CLI errors. Also, make sure that you're using the most recent AWS CLI version.

Use the AWS DMS console to create a new endpoint with extra connection attributes

1.    Open the AWS DMS console and choose the AWS Region where you want to create the endpoint.

2.    In the navigation pane, choose Endpoints.

3.    Choose Create endpoint.

4.    Choose the Endpoint type, and then enter the Endpoint configuration details.

5.    Expand the Endpoint settings section.

6.    Choose Add new setting. Use the wizard or JSON editor to provide the name and value for the endpoint.

Example:

{
  "CsvRowDelimiter": "\\n",
  "CsvDelimiter": ",",
  "CompressionType": "GZIP",
  "DataFormat": "parquet",
  "EnableStatistics": true,
  "DatePartitionEnabled": false
}

7.    (Optional) Choose Use endpoint connection attributes. Enter your extra connection attributes (ECAs). Make sure that you separate each distinct field with a semicolon.

Example:

compressionType=GZIP;csvDelimiter=,;csvRowDelimiter=\n;datePartitionEnabled=false;

8.    Expand the Test endpoint connection section.

9.    Choose your VPC and Replication instance.

9.    Choose Run test.

10.    Choose Create endpoint.

Use the AWS DMS console to add endpoint settings (or ECAs) to an existing endpoint

  1. Open the AWS DMS console, and then select the AWS Region that the endpoint is in.
  2. From the navigation pane, choose Endpoints, and then select the endpoint that you want to modify.
  3. Choose Actions, and then choose Modify.
  4. Expand the Endpoint settings section, and then use the wizard or JSON editor to enter the endpoint setting Key-Values.
  5. Choose Save.
  6. Select the endpoint again.
  7. Choose Actions, and then choose Test connection.
  8. Select your Replication instance, and then choose Run test.

Use the AWS CLI to create a new endpoint with custom endpoint settings

1.    Install and configure the AWS CLI. Select the AWS Region where you want to create the endpoint.

2.    Run the create-endpoint command:

aws dms create-endpoint --endpoint-identifier <value> --endpoint-type <value> --engine-name <value> --s3-settings ServiceAccessRoleArn=arn:aws:iam::1111111111111:role/<role-name>,BucketName=<bucket-name>,CompressionType=GZIP,MaxFileSize=32000

3.    Test the connectivity of the new endpoint with the replication instance:

aws dms test-connection--replication-instance-arn <value>
--endpoint-arn <value>

Use the AWS CLI to add endpoint settings to an existing endpoint

1.    Install and configure the AWS CLI. Select the AWS Region where you want to create the endpoint.

2.    Run the modify-endpoint command:

aws dms modify-endpoint--endpoint-arn <value>
--s3-settings Rfc4180=true,AddColumnName=true,CdcInsertsOnly=true,MaxFileSize=64000

3.    Test the connectivity of the modified endpoint with the replication instance:

aws dms test-connection--replication-instance-arn <value>
--endpoint-arn <value>

Use AWS CloudFormation to create a new endpoint with custom endpoint settings

1.    Open the CloudFormation console. Select the AWS Region where you want to create the endpoint.

2.    Choose Create stack

3.    Select Template is ready.

4.    Choose Upload a template file.

5.    Create a YAML or JSON CloudFormation file.

Example:

AWSTemplateFormatVersion: 2010-09-09
Description: "S3 target Endpoint"
Resources:
     S3TargetEndpoint:
        Type: AWS::DMS::Endpoint
        Properties:
            EndpointIdentifier: s3-Target-endpoint
            EndpointType: target
            EngineName: s3
            S3Settings:
                ServiceAccessRoleArn: arn:aws:iam::<AWS-account-number>:role/<role-name>
                BucketName: <bucket-name>
                AddColumnName: true
                CdcInsertsOnly: true
                MaxFileSize: 32000

6.    Choose Next.

7.    Enter a name for the stack.

8.    Assign an IAM role with the required permissions to create DMS resources

9.    Choose Next.

10.    Choose Submit.

Use AWS CloudFormation to add endpoint settings to an existing endpoint

1.    Open the CloudFormation console. Select the AWS Region where you want to create the endpoint.

2.    Select an existing stack, and then choose Update.

3.    On the Update stack page, choose Replace current template.

4.    Under Specify template, choose Upload a template file. Use this example template:

AWSTemplateFormatVersion: 2010-09-09
Description: "Endpoint S3 target"
Resources:
     BasicS3TargetEndpoint:
        Type: AWS::DMS::Endpoint
        Properties:
            EndpointIdentifier: s3-Target-endpoint
            EndpointType: target
            EngineName: s3
            S3Settings:
                ServiceAccessRoleArn: arn:aws:iam::<AWS-account-number>:role/<role-name>
                BucketName: <bucket-name>
                AddColumnName: true
                CdcInsertsAndUpdates: true
                MaxFileSize: 64000

5.    Choose Next.

6.    On the Specify stack details page, choose Next.

7.    (Optional) On the Configure stack options page, under Tags, add tags. Under Permissions, select the IAM role that you want to make updates on your behalf. Then, choose Next.

8.    Choose Submit.

Legacy Extra Connection Attributes

You can use the --extra-connection-attributes option to create and modify the endpoint. With extra connection attributes, you don't need to explicitly state the type of endpoint.

Example:

aws dms create-endpoint
--endpoint-identifier <value>
--endpoint-type <value>
--engine-name <value>
--username <value>
--password <value>
--server-name <value>
--port <value>
--extra-connection-attributes encryptionMode=SSE_KMS;maxFileSize=512


aws dms modify-endpoint
--endpoint-arn <value>
--extra-connection-attributes encryptionMode=SSE_KMS;maxFileSize=512

Related information

Working with AWS DMS endpoints

AWS::DMS::Endpoint

AWS UFFICIALE
AWS UFFICIALEAggiornata 6 mesi fa