Error when trying to parse OpenTelemetry 0.7.0 data from metric streams

0

I'm trying to parse open telemetry data from metrics stream and sent to s3 bucket, without any compressions.

I downloaded the files from s3 (binary), they look like

¹—
µ—
Ô

cloud.provider
aws
"
cloud.account.id
935705392901

cloud.region
	us-west-2
x
aws.exporter.arnd
...........// ARN...
ê

	NamespaceAWS/NetworkELB


MetricNameHealthyHostCount
=

hex:

00000000: aec2 020a aac2 020a d401 0a17 0a0e 636c  ..............cl
00000010: 6f75 642e 7072 6f76 6964 6572 1205 0a03  oud.provider....
00000020: 6177 730a 220a 1063 6c6f 7564 2e61 6363  aws."..cloud.acc
// skipped
00000050: 2e72 6567 696f 6e12 0b0a 0975 732d 7765  .region....us-we
00000060: 7374 2d32 0a78 0a10 6177 732e 6578 706f  st-2.x..aws.expo
00000070: 7274 6572 2e61 726e 1264 0a62 6172 6e3a  rter.arn.d.barn:
// skipped
000000e0: c002 12e7 020a 2f61 6d61 7a6f 6e61 7773  ....../amazonaws
000000f0: 2e63 6f6d 2f41 5753 2f4e 6574 776f 726b  .com/AWS/Network
00000100: 454c 422f 556e 4865 616c 7468 7948 6f73  ELB/UnHealthyHos
...

and I followed the doc https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-metric-streams-formats-opentelemetry-parse.html trying to parse it using javascript, but I keep getting an error

Error: Unknown base64 encoding at char: ¹

from this line in the sample code from the doc

const reader = new pb.BinaryReader(data)

the google-protobuf I used is 3.19.3, and metrics_service_pb is generated from proto that from the doc as well. Does anyone know how to properly parse the binary data with javascript?

Thanks, Bill

bchung
asked 2 years ago387 views
1 Answer
0

solved by converting the file from s3 to Uint8Array - example code

fs.open(filename, 'r', function (err, fd) {
    var buffer = Buffer.alloc(1);
    var data = new Uint8Array(fileSize);
    var n = 0
    while (true) {
        var num = fs.readSync(fd, buffer, 0, 1, null);
        if (num === 0)
            break;
        data[n] = buffer[0]
        n += 1
    }
    console.log(parseRecord(data));
});

not optimized, but working for me. where filename is the filename from s3, and fileSize is the num of bytes of the file.

HTH for anyone is using open telemetry

bchung
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions