EC2 instance out of memory, trying to find root cause of memory error

Question

We have a Node application running on an EC2 instance that spins up a process for each CPU core on the instance. Lately the instance keeps running out of memory and killing the process. The redundant system work perfectly fine, meaning that it spins up another instance and the load balancer directs traffic towards it when the instance becomes unresponsive.

The problem is that we are now trying to discover what cause the instance to run out of memory in the first place, so that we may optimize our code and prevent it from happening in the first place. Unfortunately we cannot check the logs for this because of log suppression. We have tried turning this off, but to no avail, we still end up with a gap in our logs from when the instance was unresponsive.

Any ideas how we might find out what in our application is causing the instance to run out of memory?

Answer

I'd suggest you to consider installing cloudwatch agent, which would continuously keep sending the logs to cloudwatch.

Here are the references to AWS Documentation for same:

- [Installing the CloudWatch agent](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-on-EC2-Instance.html)

- [Metrics collected by the CloudWatch agent](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/metrics-collected-by-CloudWatch-agent.html)

EC2 instance out of memory, trying to find root cause of memory error

Relevant content