OpenTelemetry with Elastic Beanstalk

0

Hi all

I'd like to set up AWS Open Distro for OpenTelemetry for Java with Elasic Beanstalk (https://aws-otel.github.io/docs/getting-started/java-sdk/trace-auto-instr). I've added the Java agent (using java -javaagent:path/to/aws-opentelemetry-agent.jar -jar myapp.jar). Starting the application, it logs a line like this every 15 seconds:

[otel.javaagent 2022-06-08 19:21:03:940 +0000] [OkHttp http://localhost:4317/...] ERROR io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter - Failed to export spans. The request could not be executed. Full error message: Failed to connect to localhost/127.0.0.1:4317

So I guess the ADOT Collector (https://aws-otel.github.io/docs/getting-started/collector) isn't running.

When I log into my instances (platform: Corretto 11 running on 64bit Amazon Linux 2/3.2.15), I can see the CloudWatch Agent under /opt/aws/amazon-cloudwatch-agent (version: 1.247350.0). It includes the Collector under /opt/aws/amazon-cloudwatch-agent/cwagent-otel-collector:

$ pwd
/opt/aws/amazon-cloudwatch-agent
$ ls
LICENSE  NOTICE  RELEASE_NOTES  THIRD-PARTY-LICENSES  bin  cwagent-otel-collector  doc  etc  logs  var
$ ls cwagent-otel-collector/
etc  logs  var
$ ls bin
CWAGENT_VERSION          amazon-cloudwatch-agent-config-wizard  config-downloader  cwagent-otel-collector
amazon-cloudwatch-agent  amazon-cloudwatch-agent-ctl            config-translator  start-amazon-cloudwatch-agent

I try to start the OTel Collector using sudo bin/amazon-cloudwatch-agent-ctl -a start -m ec2 -o default. It doesn't start, though. Here's the process status:

$ sudo bin/amazon-cloudwatch-agent-ctl -a status
{
  "status": "running",
  "starttime": "2022-06-09T07:12:30+0000",
  "configstatus": "configured",
  "cwoc_status": "stopped",
  "cwoc_starttime": "",
  "cwoc_configstatus": "configured",
  "version": "1.247350.0"
}

And the log:

$ tail cwagent-otel-collector/logs/cwagent-otel-collector.log
2022/06/09 07:09:20 E! User aoc does not exist: no matching entries in passwd file
2022/06/09 07:09:20 E! Failed to ChangeUser: no matching entries in passwd file
2022/06/09 07:09:20 no matching entries in passwd file

Has anyone gotten the OTel Collector working with Elastic Beanstalk? In tips for how to configure this?

-martin

1 Answer
0
Accepted Answer

I filed an issue on the Amazon Cloudwatch Agent repo on GitHub: https://github.com/aws/amazon-cloudwatch-agent/issues/483

Seems the problem is fixed with the latest release of the agent.

answered 2 years ago
  • I see that the team had replied back to you on the github issue and when you manually installed the latest version it worked. I will suggest you to use ebextensions [1] to install the latest version of the agent so that you don't have to manually install the latest version on every instance.

    Reference :

    [1] Advanced environment customization with configuration files (<code>.ebextensions</code>) - https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions.html

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions