Publish custom device defender metrics from lambda

Question

We have a fleet of devices that publish runtime metrics to a custom IoT MQTT topic.  The metrics are published as a binary data blob to reduce data cost for our fleet of devices.  Publishes to the MQTT topic trigger a lambda which decode the data and send them to our backend.

We are interested in ingesting these runtime stats into Device Defender as custom metrics.  We want to have the lambda that is currently being triggered also publish the metrics to device defender so our devices don't have to push the same information (once to our existing MQTT topic and once to device defender as a JSON blob) and incur more than double the data cost.

I see there is no HTTP API for publishing metrics into Device Defender and am wondering if this use case is supported (e.g. having a lambda publish metrics to device defender).  Looking for any guidance or thoughts here.

We have tried setting up the lambda to do what is described above but are getting the following errors when attempting to publish to the device defender topic:
```
InvalidRequestException: Invalid publish to restricted topic using HTTP
```

We are using the AWS Go client which I believe is using an HTTP post to publish MQTT messages under the hood.  I see there is no HTTP API for publishing metrics into Device Defender directly and am wondering if this use case is supported (e.g. having a lambda publish metrics to device defender).  Looking for any guidance or thoughts here.

Answer

Hello,

Thank you for connecting and sharing your use case.

The IoT Device Defender custom metrics are generally emitted by the Device defender agent which is running on the device. Ideally, you can emit the custom metric from the Lambda function, if you can ingest the data from the device to the Lambda environment and manage to implement a version of the SDK [1]. You should be able to emit the required custom metric for your use case.

The specific error that you are seeing is likely due to attempting to publish on a restricted topic which doesn't support publish-in from the Lambda environment or rather any clients.

Please feel free to explore the agent SDK and share if that fits your use case.

[1] https://github.com/aws-samples/aws-iot-device-defender-agent-sdk-python

Answer

The device defender topics only support MQTT publish so you will need to have your Lambda function connect over MQTT to publish those metrics to the device defender reserved topics.

This means that you will still incur the MQTT message cost from Lambda so whether you publish it from the device or Lambda, the cost may not change.

See below excerpts from - https://docs.aws.amazon.com/iot/latest/developerguide/reserved-topics.html#reserved-topics-device-defender

“These messages support response buffers in Concise Binary Object Representation (CBOR) format and JavaScript Object Notation (JSON), depending on the payload-format of the topic. AWS IoT Device Defender topics only support MQTT publish.”

Publish custom device defender metrics from lambda

Relevant content