Large MQTT OTA file transfer fails at block 1024

Question

I am working on setting up the freeRTOS OTA example in MQTT mode, modified to transfer large files (> 10MB).
I have modified `pOtaInterfaces` to write my files to SD using fseek() and fwrite().
On a smaller file, this works perfectly, and I receive a complete file.
On large files, however, when the transfer gets to block 1024, I start to get `Received a duplicate block` and the block numbers seem to be muddled. It jumps from 1023 to 1025, then 1027, then 1056.
It looks very much like something in the library is corrupting bit 10 of whatever system keeps track of the blocks. I suppose it is the bitmap system.
At this point the transfer slows down hugely, and if I wait for it to complete, I have a corrupted file.
Does anyone know this OTA bitmap system well enough to help me identify if there is indeed a bug in the SDK?

Accepted Answer

Hi Andrew. Please check the value of `OTA_MAX_BLOCK_BITMAP_SIZE` in your project. I suspect it's 128. That means 128 bytes, or 1024 bits, to track the blocks. If so, you will need to increase this value to have enough bits for your number of blocks.

- https://github.com/FreeRTOS/FreeRTOS/blob/391c79958f635ee5476dcf2774dab59e2b151eff/FreeRTOS-Plus/Demo/AWS/Ota_Windows_Simulator/Ota_Over_Mqtt_Demo/DemoTasks/OtaOverMqttDemoExample.c#L494
- https://github.com/aws/aws-iot-device-sdk-embedded-C/blob/c70269486b3fdcb6d6e85e999059d0bd35e732cd/demos/ota/ota_demo_core_mqtt/ota_demo_core_mqtt.c#L330

You should review all of these settings and tailor to your needs: https://github.com/aws/ota-for-aws-iot-embedded-sdk/blob/fb4aece8c7ee4211b17ee48196222c04795b7a51/source/include/ota_private.h#L62-L70

More information here (although it doesn't cover the bitmap setting): https://www.freertos.org/Documentation/api-ref/ota-for-aws-iot-embedded-sdk/docs/doxygen/output/html/ota_config.html

As a side point, does your device include a HTTP client? With such a large file, your MQTT messaging costs will likely rival your job action costs. You may find HTTP file transfer to be more cost effective.

Large MQTT OTA file transfer fails at block 1024

Relevant content