Is there an option to persist MQTT offline messages to disk?

Question

Hi All,  
  
I am using the Python AWS IoT MQTT library.  I see that it has an internal offline message queue which can store messages if the connection to the broker is down.  Is there an option in the library to also save that queue to disk so the items in the queue would survive a reboot or power-cycle event?  
  
Thank you.  
  
Edited by: Seanlabs on Jun 25, 2020 4:54 PM

Accepted Answer

This kind of thing is good to build in your own application, rather than the SDK, since everyone's going to have different desires for how this kind of thing might work. Have it store the QoS1 message to disk/database/etc before passing the message to the MQTT client, then remove the message from storage when the puback (acknowledgement from server) is received. When your application starts up, iterate over any stored messages and send them to the just-started MQTT client.  
  
In the V1 SDK: https://github.com/aws/aws-iot-device-sdk-python  
you can use the publishAsync():  
https://github.com/aws/aws-iot-device-sdk-python/blob/0fb8d7859016386c0aa5dbd8e8aa07da8f187505/AWSIoTPythonSDK/MQTTLib.py#L630  
and pass an ack callback to know when the server has acknowledged the message.  
  
In the V2 SDK:  
https://github.com/aws/aws-iot-device-sdk-python-v2  
the publish() call returns a tuple, where the 1st item is a future that completes when the puback is received:  
https://awslabs.github.io/aws-crt-python/api/mqtt.html#awscrt.mqtt.Connection.publish  
so you'd use future.add_done_callback():  
https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future.add_done_callback  
to know when the server has acknowledged the message

Answer

Hello,  
  
Thanks for trying the IoT SDK for Python.  Unfortunately, there is no current out-of-the-box way to persist offline operations to disk.  The v1 Iot SDK for Python is in maintenance mode (i.e. no new features planned, merging high priority bug/security fixes and potentially small features only) in favor of the new v2 Iot Sdk For Python (https://github.com/aws/aws-iot-device-sdk-python-v2).  Full disclosure however, the v2 SDK does not support this either.  
  
That being said, I think a small pull request that allowed the user to inject a custom offline queue (see the default OfflineRequestsManager at https://github.com/aws/aws-iot-device-sdk-python/blob/master/AWSIoTPythonSDK/core/protocol/internal/workers.py#L279) would not have trouble getting approved if you wanted to give it a try.

Is there an option to persist MQTT offline messages to disk?

Relevant content