To collect traces from DAPR we are using Opentelemetry collector and using Zipkin receiver since that's supported by DAPR and using awsxray exporter in open telemetry configuration. We see the traces generated in Opentelemetry log, however we do not see traces collected in AWS Xray. Please let us know what could be the issue
Open telemetry configuration:
receivers:
zipkin:
endpoint: 0.0.0.0:9411
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
awsxray:
awsxray/udp_endpoint:
endpoint: "0.0.0.0:5678"
transport: udp
awsxray/proxy_server:
proxy_server:
endpoint: "0.0.0.0:1234"
tls:
insecure: true
region: ap-south-1
Exporter configuration:
awsxray:
region: ap-south-1
index_all_attributes: true
resource_arn: "arn:aws:iam::215977048474:role/EKS-NodeGroup"
#role_arn: "arn:aws:iam::215977048474:role/MastBazaar-EKS"
no_verify_ssl: true
Service configuration:
service:
extensions: [pprof, zpages, health_check]
pipelines:
traces:
receivers: [zipkin]
# List your exporter here.
exporters: [awsxray,logging]
Also added environment variable OTEL_PROPAGATORS: "xray" in deployment as seen below
containers:
- name: otel-collector
image: otel/opentelemetry-collector-contrib-dev:latest
env:
- name: OTEL_PROPAGATORS
value: "xray"
Sample log in open telemetry
2022-09-13T18:15:46.106Z INFO loggingexporter/logging_exporter.go:42 TracesExporter {"#spans": 1}
2022-09-13T18:15:46.106Z DEBUG loggingexporter/logging_exporter.go:51 ResourceSpans #0
Resource SchemaURL:
Resource labels:
-> service.name: STRING(multiplyapp)
ScopeSpans #0
ScopeSpans SchemaURL:
InstrumentationScope
Span #0
Trace ID : 5db6a8eb0ade84a251ca547034b13c5f
Parent ID : 9987294085d81f24
ID : f88d24f02301bf28
Name : CallLocal/multiplyapp/multiply
Kind : SPAN_KIND_SERVER
Start time : 2022-09-13 18:15:44.031751 +0000 UTC
End time : 2022-09-13 18:15:45.034974 +0000 UTC
Status code : STATUS_CODE_UNSET
Status message :
Attributes:
-> dapr.api: STRING(/dapr.proto.internals.v1.ServiceInvocation/CallLocal)
-> dapr.invoke_method: STRING(multiply)
-> dapr.protocol: STRING(grpc)
-> rpc.service: STRING(ServiceInvocation)
-> net.host.ip: STRING(10.0.195.115)
2022-09-13T18:15:54.505Z INFO loggingexporter/logging_exporter.go:42 TracesExporter {"#spans": 1}
2022-09-13T18:15:54.505Z DEBUG loggingexporter/logging_exporter.go:51 ResourceSpans #0
Resource SchemaURL:
Resource labels:
-> service.name: STRING(multiplyapp)
ScopeSpans #0
ScopeSpans SchemaURL:
InstrumentationScope
Span #0
Trace ID : c3f98c14589d4618899f4581503e4c3f
Parent ID : 642dd665bbc6c3dd
ID : 014fec0dfb862b5c
Name : CallLocal/multiplyapp/multiply
Kind : SPAN_KIND_SERVER
Start time : 2022-09-13 18:15:53.489635 +0000 UTC
End time : 2022-09-13 18:15:54.492579 +0000 UTC
Status code : STATUS_CODE_UNSET
Status message :
Attributes:
-> dapr.api: STRING(/dapr.proto.internals.v1.ServiceInvocation/CallLocal)
-> dapr.invoke_method: STRING(multiply)
-> dapr.protocol: STRING(grpc)
-> rpc.service: STRING(ServiceInvocation)
-> net.host.ip: STRING(10.0.195.115)
Any guidance will be very much appreciated
