How can I solve the "unexpected namespace: local" xray issue?

Question

I'm running an xray daemon on an ec2-backed ecs cluster, and I'm getting this in the xray daemon's cloudwatch logs:

> warn awsxrayreceiver@v0.78.0/receiver.go:116 X-Ray segment to OT traces conversion failed {"kind": "receiver", "name": "awsxray", "data_type": "traces", "error": "unexpected namespace: local"}

This is somehow caused by a python flask app instrumented with xray. There must be some configuration that I'm missing for this? Maybe I'm also missing a configuration in xray somewhere too?

What environment variable or other configuration change can I make to fix this problem?

In case it is interesting, I do see traces and trace maps despite the warnings.

Python code:

requirements:
```
aws-xray-sdk
```

app.py:
```
from aws_xray_sdk.core import xray_recorder  # type: ignore
from aws_xray_sdk.ext.flask.middleware import XRayMiddleware  # type: ignore
...
app = Flask(__name__)
...
xray_recorder.configure(service="helloworld")
XRayMiddleware(app, xray_recorder)
```

environment variables:
```
AWS_XRAY_DAEMON_ADDRESS="172.17.0.1:2000"
```

For the xray daemon, I am using `public.ecr.aws/aws-observability/aws-otel-collector:v0.30.0` as the image, and no environment variables or command overrides.

re:Post Agent · Answer

The "unexpected namespace: local" issue you're encountering is likely due to a mismatch between the X-Ray SDK configuration and the expected format for the X-Ray daemon.

To resolve this issue, you should try the following:

1. Update your Python code to explicitly set the namespace when configuring the X-Ray recorder. Modify your app.py as follows:

```python
from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.ext.flask.middleware import XRayMiddleware

app = Flask(__name__)

xray_recorder.configure(service="helloworld", daemon_address="172.17.0.1:2000", context_missing='LOG_ERROR', namespace='aws')
XRayMiddleware(app, xray_recorder)
```

By adding the `namespace='aws'` parameter, you're explicitly setting the namespace to 'aws', which is one of the supported namespaces for X-Ray.

2. Ensure that your X-Ray daemon configuration is correct. If you're using the AWS Distro for OpenTelemetry (ADOT) collector as your X-Ray daemon, make sure you have the proper configuration for the X-Ray receiver. You might need to add a configuration file for the ADOT collector that includes the X-Ray receiver settings.

3. Set the `AWS_XRAY_CONTEXT_MISSING` environment variable to `LOG_ERROR` in your ECS task definition. This will prevent the X-Ray recorder from raising runtime errors if there's no active segment.

```
AWS_XRAY_CONTEXT_MISSING=LOG_ERROR
```

4. Double-check that your ECS task role has the necessary permissions to send traces to X-Ray.

5. If you're still seeing issues, you might want to increase the log level for the X-Ray SDK to get more detailed information. You can do this by adding the following to your Python code:

```python
from aws_xray_sdk.core import xray_recorder
import logging

logging.getLogger("aws_xray_sdk").setLevel(logging.DEBUG)
xray_recorder.configure(service="helloworld", daemon_address="172.17.0.1:2000", context_missing='LOG_ERROR', namespace='aws')
```

These changes should help resolve the "unexpected namespace: local" issue. If you're still seeing traces and trace maps despite the warnings, it suggests that some data is being successfully sent to X-Ray. However, implementing these changes should eliminate the warnings and ensure all your trace data is being properly processed.  
**Sources**  
[Django — aws-xray-sdk 2.13.1 documentation](https://docs.aws.amazon.com/xray-sdk-for-python/latest/reference/frameworks.html)  
[Configuring the AWS X-Ray daemon - AWS X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-configuration.html)  
[Third Party Library Support — aws-xray-sdk 2.13.1 documentation](https://docs.aws.amazon.com/xray-sdk-for-python/latest/reference/thirdparty.html)

How can I solve the "unexpected namespace: local" xray issue?

Add your answer

Relevant content