Failed connect two tasks in ECS Fargate

Question

context:

I have 2 microservices service A and service B
service A is a general nestjs microservice while service B is using nestjs + gRPC
service A calls service B API
local test:

local: service A and B are brought up using separate docker-compose.yml and I set them explicitly to use the same network. Everything works as expected.
PROBLEM: I deployed both services to AWS ECS Fargate. Each of them has an individual task definition. I setup service discovery for service B (A type) and the Service discovery endpoint is serviceb.local. Currently connection between service A and service B are not working. I keep receiving ERROR 14 connection not established error.

QUESTION: How do i make containers in two different tasks talk to one another.

Code:

register Service B's client in Service A
```
@Global()
@Module({
  imports: [
    ClientsModule.register([
      {
        name: SERVICE_B_NAME,
        transport: Transport.GRPC,
        options: {
          url: 'serviceb.local',
          package: SERVICE_B_PACKAGE_NAME,
          protoPath: 'node_modules/services-proto/proto/serviceb.proto',
        },
      },
    ]),
  ],
```

bootstrap Service B
```
  const app: INestMicroservice = await NestFactory.createMicroservice(AppModule, {
    transport: Transport.GRPC,
    options: {
      url: '0.0.0.0:50051',
      package: protobufPackage,
      protoPath: join('node_modules/services-proto/proto/serviceb.proto'),
    },
  });
```
What I have tries:

I called service B's public IP from local postman and everything works;
I used Cloud9 IDE and verified that Service discovery endpoint serviceb.local is successfully resolved to the public IP;
Locally testing in docker works as expected;

sanju_s · Answer

Hello,

I understand that you are facing issue with connectivity between 2 ECS Services.

To test connectivity between 2 services, you can consider to perform the following:

-> Exec into the container of Task running in Service A and try reaching out to another service. Also try vice versa and if the connectivity is fine then that means both the services can reach each other and communicate as well.

-> Also you can test the connectivity for the service discovery endpoint of service B from service A

-> We can make use of dig or curl commands to  to check endpoint connectivity

This would help us to isolate the issue from networking side and further narrow down whether the issue is at application level or not. Also if our Tasks are healthy and we are using gRPC to connect 2 services, then it is also possible that error is coming from application container. However as mentioned above, the connectivity testing will help us to identify the cause of issue.

References:

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-discovery.html

Failed connect two tasks in ECS Fargate

Relevant content