Skip to content

AWS IoT Core Commands static CBOR payload is corrupted when uploading binary file

0

Hi, I would like to create an AWS IoT command that restarts the device.

The command will have the following JSON format:

{
  "cmd": "reboot"
}

Converted to CBOR, the payload has the following hexadecimal format: b9000163636d64667265626f6f74

I create a command on the web interface, I provide the file containing the CBOR, and I put the Payload file format type as application/cbor. When I execute the command, my device receives the following data: efbfbd0163636d64667265626f6f74

We observed that the b900 bytes were transformed into efbfbd, rendering the CBOR invalid. The problem appears to occur during command creation: the CBOR file is treated by the AWS IoT Commands pipeline as text content rather than raw binary data.

During this processing, the invalid UTF-8 bytes are replaced with the Unicode substitution character U+FFFD (0xEF 0xBF 0xBD), which corrupts the binary payload.

Solution is to convert the CBOR to base64, but this requires the device to decode base64, which is not optimal. I would like to be able to send binary data directly.

Note that in the case of a dynamic payload, there is a preprocessing step (AWS JSON Substitution: CBOR) which allows sending CBOR binary data directly. The problem is that it is impossible to send binary data directly with a static payload.

  • If my answer helped solve your problem, I would appreciate it if you click on “accepted answer”

asked 2 months ago55 views
2 Answers
3

To add a bit of practical context regarding the 'corruption' you're seeing:

The AWS Management Console and most API clients treat the payload.content field as a UTF-8 string by default. When you upload a raw binary CBOR file, the pipeline encounters non-UTF-8 bytes (like your 0xB9 0x00), fails to map them, and replaces them with the U+FFFD substitution character.

If you want to stick with a static payload but avoid manual Base64 handling in your code, you can use the AWS CLI with the fileb:// prefix. This tells the CLI to treat the file as binary and handle the encoding for the API call automatically:

aws iot create-command \
    --command-id "reboot-device" \
    --payload content="$(cat command.cbor | base64)",contentType="application/cbor"

However, I highly second the suggestion of using Dynamic Payloads with CBOR preprocessing. It is the most robust way to ensure that what arrives at the MQTT broker is 'clean' binary data, especially if you want to avoid adding a Base64 decoder library to a resource-constrained IoT device.

EXPERT
answered 2 months ago
1

Based on the AWS IoT Core Commands documentation, when creating a command with a static payload, the payload content is expected to be provided as base64-encoded binary data. The payload object accepts a content field that should contain base64-encoded binary data, along with a contentType field to specify the format (such as "application/cbor").

While I understand your concern about requiring devices to decode base64, this is the intended design for static payloads in AWS IoT Commands. The base64 encoding ensures that binary data is properly transmitted through the command creation and execution pipeline without corruption.

For your use case where you want to send CBOR binary data directly without base64 encoding on the device side, you have a couple of options:

  1. Use a dynamic payload with preprocessing: As you noted, dynamic payloads support the AWS JSON Substitution preprocessor with CBOR output format, which can handle binary CBOR data directly. Even for a simple static-like command, you could create a dynamic payload template without any placeholders and use the CBOR preprocessor to ensure proper binary handling.

  2. Implement base64 decoding on the device: While not optimal from your perspective, this is the supported approach for static payloads. The base64 decoding overhead is typically minimal for most IoT devices.

The behavior you're experiencing where b900 is being transformed to efbfbd (the UTF-8 replacement character) confirms that the static payload is being processed as text rather than binary when not base64-encoded. This is why the documentation recommends specifying the payload content type and using base64 encoding for binary content in static payloads.
Sources
CommandPayload - AWS IoT
create_command - botocore 1.42.50 documentation
create_command - Boto3 1.42.50 documentation

answered 2 months ago
EXPERT
reviewed 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.