Connection reset error in ECS cluster between two microservices running on two containers.

Hi,

Thank you for quick response. Really appreciate it.

Communication between microservices is standard REST based HTTP communication. We observed connection reset error frequent.

Another observation is if it's more likely to happen when the service has been idle for some time.

We are using application load balancer. We are trying to check ELB logs to see if we get anything there.

Do you think setting header "Keep-Alive" will help?

Thank You,

Vishwas

What is the communication patter looks like and is the TCP connection linger with no packet for a long period of time or the connection reset happens very often?

Also What type of ELB you are using?

For example, in NLB,

Elastic Load Balancing sets the idle timeout value for TCP flows to 350 seconds. You cannot modify this value. 

That said, connection reset should not be a big problem, especially if they don't occur often, this is because the client application can always open a new TCP connection at the error handling logic. As matter of fact, if the application is failing because of this, it indicates issues with the code in terms of gracefully handle error conditions.

As indicated in the documentation, TCP Keep-alive is a viable option. Or you can set application code to timeout before TCP connection is closed.

It should also be noted that the backend server should also have keep alive values larger than the ELB's setting.