aws xray not working with long running ecs task

0

I am using the AWS XRay SDK for Nodejs in a ECS task. This task is a very long running process that can be running for up to a couple of days. I have noticed that once done there is no xray traces available. Is there a timeout or is the xray daemon running out of memory staying alive that long?

  • Do you see anything in the Xray logs? Are you able to reproduce this issue?

asked 2 years ago557 views
1 Answer
0

Hello,

I understand that you are currently unable to get any X-Ray tracs when integrating AWS XRay SDK for Nodejs in a ECS task.

Firstly, AWS X-Ray is Application Performance Monitoring tool that can be used to trace transactions for application. Please note, AWS X-Ray supports tracing for applications that are written in Node.js [1], Java, and .NET.

To forward traces to X-Ray API, you will need the X-Ray Daemon inside your ECS task. Here [2] is the complete documentation to achieve this. The task can be configured using "bridge" or "awsvpc" network mode, both approaches explained on the aforementioned doc.

Secondly, if you are unable to get any traces, then you want to first check the X-Ray daemon logs (The daemon sends a telemetry data per minute.) from the Cloudwatch LogStream to check for any errors that maybe occurring.

There can be errors such as -

  1. X-Ray instrumentation/configuration error such as not providing service name.

https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-nodejs-configuration.html

  1. Trace Size Exceeded - which can mean that the application is attempting to send too many segments for a particular trace at a time.

https://docs.aws.amazon.com/xray/latest/devguide/xray-console-sampling.html

Additionally you can also check with our service team on Github at [3], who are actively monitoring this Repo, to report for any X-Ray daemon issues or feature requests.

Thirdly, if the suggestions above do not help resolve the issue, this will need troubleshooting based on your configurations. Could you please create a support case with our premium support team who can then check the details on your resource configurations?

Please do not post any sensitive information over re:Post since this is a public platform.

As always, feel free to reach back with any further questions or concerns in the meantime!


References:

[1] https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-nodejs-configuration.html

[2] https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-ecs.html

[3] https://github.com/aws/aws-xray-daemon/issues

profile pictureAWS
SUPPORT ENGINEER
Yash_C
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions