How to send log data from Lambda to EC2-hosted ElasticSearch

Question

I have created a lambda and set the kinesis data stream as a trigger. When I send a log to the kinesis data stream, I can see lambda being triggered and I can view the logs on cloud watch.

I have reserved an **EC2 instance and installed elastic search** and listening at 9200 port (ex say some IP like https://11.22.333.44:9200/).
Now I need to send the data from lambda to elastic search which I have installed in an EC2 instance.
When I clicked on the destination I do not have a way to reach EC2 instance hosting my Elastic search.

**Note**: I do not want to use "Amazon OpenSearch Service".

Answer

Hi,

I would take inspiration from this: https://aws.amazon.com/blogs/compute/using-aws-lambda-extensions-to-send-logs-to-custom-destinations/, which uses Lambda extensions, and perform http request against the elastic endpoint hosted in ec2.

Hope it helps ;)

Answer

As your Lambda logs are in CloudWatch, you can install the Elastic Agent on your EC2 and use the ElasticSearch CloudWatch integration, specifying the LogGroup prefixes to pull.

See https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html and https://docs.elastic.co/en/integrations/aws.

How to send log data from Lambda to EC2-hosted ElasticSearch

Relevant content