ECS Service connect could not resolve host

0

Hello. I've been looking into using ECS Service connect for one of the projects. I've configured ECS service with Service connect functionality but when I try to connect to it from other service in the same namespace I get curl: (6) Could not resolve host: core

Service configuration

"serviceConnectConfiguration": {
    "enabled": true,
    "namespace": "arn:aws:servicediscovery:us-east-1:***:namespace/ns-awd3wsdifs2nsrla",
    "services": [
        {
            "portName": "core",
            "discoveryName": "core",
            "clientAliases": [
                {
                    "port": 80,
                    "dnsName": "core"
                }
            ]
        }
    ]
},
"serviceConnectResources": [
    {
        "discoveryName": "core",
        "discoveryArn": "arn:aws:servicediscovery:us-east-1:***:service/srv-rirfgcxi4bpplqyj"
    }
]

Curl results

curl -v http://core:80
* Could not resolve host: core
* Closing connection 0
curl: (6) Could not resolve host: core

curl -v http://core.skc:80
* Could not resolve host: core.skc
* Closing connection 0
curl: (6) Could not resolve host: core.skc

Source service config

"serviceConnectConfiguration": {
    "enabled": true,
    "namespace": "arn:aws:servicediscovery:us-east-1:***:namespace/ns-awd3wsdifs2nsrla",
    "services": [
        {
            "portName": "admin",
            "discoveryName": "admin",
            "clientAliases": [
                {
                    "port": 77,
                    "dnsName": "admin"
                }
            ]
        }
    ]
},
"serviceConnectResources": [
    {
        "discoveryName": "admin",
        "discoveryArn": "arn:aws:servicediscovery:us-east-1:***:service/srv-niqcn3ec7jgsz27i"
    }
]

VPC has DNS and DNS hostnames enabled

{
    "VpcId": "vpc-00bdb82e8d17a444a",
    "EnableDnsSupport": {
        "Value": true
    }
}

{
    "VpcId": "vpc-00bdb82e8d17a444a",
    "EnableDnsHostnames": {
        "Value": true
    }
}

What am I missing? Thank you.

asked a year ago4332 views
2 Answers
0
Accepted Answer

Actually, figured it out.

Sequence of events:

  1. Deploy service1
  2. Deploy service2
  3. Connect to container in ECS service1 and try curl http://service2:80 - this results in could not resolve host
  4. Connect to container in ECS service2 and try curl http://service1:80 - it works
  5. Restart service1 with --force-new-deployment and repeat step 3 - and now it works.

Apparently, ECS adds those ECS Service connect related records in /etc/hosts. Line Service Connect configures DNS names for your services in the task itself, and doesn't require nor create DNS records in your hosted zones. suggests this.

/etc/hosts on service1 in step 3

127.0.0.1 localhost
10.0.20.61 ip-10-0-20-61.eu-west-2.compute.internal
127.255.0.1 service1
2600:f0f0:0:0:0:0:0:1 service1

/etc/hosts on service2 in step 4

127.0.0.1 localhost
10.0.20.167 ip-10-0-20-167.eu-west-2.compute.internal
127.255.0.1 service1
2600:f0f0:0:0:0:0:0:1 service1
127.255.0.2 service2
2600:f0f0:0:0:0:0:0:2 service2

/etc/hosts on service1 in step 5

127.0.0.1 localhost
10.0.20.13 ip-10-0-20-13.eu-west-2.compute.internal
127.255.0.1 service1
2600:f0f0:0:0:0:0:0:1 service1
127.255.0.2 service2
2600:f0f0:0:0:0:0:0:2 service2

Looks like ECS does not update /etc/hosts dynamically when new endpoints are added into CloudMap namespace.

answered a year ago
0

I know this is an old post but there is not much information out there so I thought I would append more information to this solution. I was encountering a similar issue using CloudFormation templates where one service was a client-server service and the others were client only. After stumbling upon this answer, I confirmed it worked in my instance. In order to ensure that it works without manual input, I changed the CloudFormation template to ensure that the client services were "DependsOn" with the client-server service. This allowed the Service Connect sidecar to get properly running since it updates the /etc/hosts file in the running container of all Service Connect services. Now, the client service has the values in the host file when it running.

Chris
answered 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions