How to get "client-less" traces in X-Ray's service map

0

Hi,

I've been trying to make a bit more sense of my X-Ray traces, and I've been seeing a lot of stuff that is initated by my service (like initialisation calls or stuff done in an update loop) being seen by X-ray as having a client request behind them.

Traces are sent from a go program with the otel go SDK to an OTEL collector which then forwards them to the AWS account. I've tried intialising the trace with different traceKind (Internal, Producer) to no avail.

I was able to get what I wanted using the aws client to send out edited traces with an invalid 'parent_id' parameter to get the visualisation I wanted in the ServiceMap, but this is pretty hacky & I couldn't even get the same result through OTEL's go sdk.

Here's a crudely illustrated example of what I have & what I want to achieve.

(The service in the example is actually run locally for testing, I just forced the origin to AppRunner so the serviceMap doesn't get confused & so I do not confuse them with actual ECS instances of the services that are running)

For the sake of example, here's a trace with an "unwanted client", and the manually edited version which appears "clientless" in the service map:

The version with the client link

{
    "Id": "1-63ee407a-35043ec3121f735baa80ebea",
    "Duration": 0.801,
    "LimitExceeded": false,
    "Segments": [
        {
            "Id": "497e0cc431330383",
            "Document": {
                "id": "497e0cc431330383",
                "name": "SQS",
                "start_time": 1676481266.4775069,
                "trace_id": "1-63ee407a-35043ec3121f735baa80ebea",
                "end_time": 1676481267.278352,
                "fault": false,
                "error": false,
                "throttle": false,
                "aws": {
                    "queue_url": "https://sqs.eu-west-1.amazonaws.com/639648953767/rt-netsample-ServicesStack-VpcDev-snsrtnetsampledevonlinebmatchmaki-rRQaezXx98aB",
                    "xray": {
                        "auto_instrumentation": false
                    },
                    "region": "eu-west-1",
                    "operation": "SendMessageBatch",
                    "request_id": "618bb528-1ab6-5fe3-8fc0-e94f0f854783"
                },
                "metadata": {
                    "default": {
                        "messaging.system": "AmazonSQS",
                        "messaging.url": "https://sqs.eu-west-1.amazonaws.com/639648953767/rt-netsample-ServicesStack-VpcDev-snsrtnetsampledevonlinebmatchmaki-rRQaezXx98aB",
                        "aws.service": "SQS",
                        "otel.resource.cloud.platform": "aws_app_runner",
                        "otel.resource.cloud.provider": "aws"
                    }
                },
                "origin": "AWS::AppRunner::Service"
            }
        }
    ]
}

The version without:

{
    "Id": "1-63ee41c9-a06f9285771aedcd36b04dd3",
    "Duration": 0.801,
    "LimitExceeded": false,
    "Segments": [
        {
            "Id": "497e0cc431330383",
            "Document": {
                "id": "497e0cc431330383",
                "name": "SQS",
                "start_time": 1676481266.4775069,
                "trace_id": "1-63ee41c9-a06f9285771aedcd36b04dd3",
                "end_time": 1676481267.278352,
                "parent_id": "004f72be19cddc2b",
                "fault": false,
                "error": false,
                "throttle": false,
                "aws": {
                    "queue_url": "https://sqs.eu-west-1.amazonaws.com/639648953767/rt-netsample-ServicesStack-VpcDev-snsrtnetsampledevonlinebmatchmaki-rRQaezXx98aB",
                    "xray": {
                        "auto_instrumentation": false
                    },
                    "region": "eu-west-1",
                    "operation": "SendMessageBatch",
                    "request_id": "618bb528-1ab6-5fe3-8fc0-e94f0f854783"
                },
                "metadata": {
                    "default": {
                        "messaging.system": "AmazonSQS",
                        "messaging.url": "https://sqs.eu-west-1.amazonaws.com/639648953767/rt-netsample-ServicesStack-VpcDev-snsrtnetsampledevonlinebmatchmaki-rRQaezXx98aB",
                        "aws.service": "SQS",
                        "otel.resource.cloud.platform": "aws_app_runner",
                        "otel.resource.cloud.provider": "aws"
                    }
                },
                "origin": "AWS::AppRunner::Service"
            }
        }
    ]
}

Thanks in advance !

asked a year ago96 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions