DMS Test Endpoint failed

0

I am trying to create a DMS replication task with an RDS Postgresql source. The endpoint connection is failing with the following message:

Test Endpoint failed: Application-Status: 1020912, Application-Message: Failed to build connection string Unable to find Secrets Manager secret, Application-Detailed-Message: Failed to retrieve secret. Unable to find AWS Secrets Manager secret Arn 'arn:aws:secretsmanager:<region>:<account>:secret:<secret>' The secrets_manager get secret value failed: curlCode: 28, Timeout was reached Too many retries: curlCode: 28, Timeout was reached

I checked that the secret ARN is correct. I have also set "SecretsManagerAccessRoleArn" for the endpoint, which I double-checked. This role has the following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "secretsmanager:DescribeSecret",
                "secretsmanager:GetSecretValue"
            ],
            "Resource": [
                "arn:aws:secretsmanager:<region>:<account>:secret:<friendly-name>-??????"
            ],
            "Effect": "Allow"
        },
        {
            "Action": [
                "kms:Decrypt",
                "kms:DescribeKey"
            ],
            "Resource": [
                "arn:aws:kms:<region>:<account>:key/*"
            ],
            "Effect": "Allow"
        }
    ]
}

The secretsmanager resource matches the secret ARN. I am using the default encryption key, so I believe explicit kms permission is not necessary. I just added it out of desperation. Here is the role trust policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "dms.amazonaws.com",
                    "dms.<region>.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

According to the documentation, the region-specific principle should be used; I tried adding dms.amazonaws.com when it didn’t work.

The replication instance is on a public subnet. I tried aws secretsmanager get-secret-value from another instance on the same subnet using the SecretsManagerAccessRole as assumed-role and it works.

The roles, policies, and dms resources are all instantiated via cloudformation.

Any help getting this to work would be much appreciated.

1 Answer
0
Accepted Answer

The error Timeout was reached points to a connectivity problem more than a permission problem. I would check that the DMS instance has connectivity to Secrets Manager API.

  1. If possible, check if you can create a VPC Endpoint for SSM https://docs.aws.amazon.com/secretsmanager/latest/userguide/vpc-endpoint-overview.html
  2. Otherwise, make sure the Instance has Internet Access. As a DMS instance it should probably live in a Private Subnet having Internet Access through a NAT Gateway https://aws.amazon.com/premiumsupport/knowledge-center/nat-gateway-vpc-private-subnet/

Hopefully this helps you!

AWS
answered 2 years ago
  • Thank you. This occurred to me. I have "PubliclyAccessible": false set on the replication instance, perhaps that is why. I created a VPC endpoint for secretsmanager on the same subnet. I have DNS enabled on the VPC and the VPC endpoint, which I thought would suffice, but it did not. I recreated the instance with secretsManagerEndpointOverride per the dms documentation, still failing with timeout.

  • Security group on the VPC endpoint wasn’t correct. Now it is working.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions