Sending high throughput messages to IOT devices

0

We've been using a single backend app that connected via MQTT to listen for devices messages and send replies. After hitting the 100msg/sec limit we have switched to the HTTPS action to receive mqtt messages and we're still using a single MQTT connection to send replies.

However if the replies to be sent exceed the 100msg/sec limit of a single connection, what's the best way to go past that limit? We would like to avoid having to create multiple MQTT connections which seems to be just a workaround rather than a well thought solution.

Should we use the https data endpoint? Or this API?

alex88
已提问 1 年前259 查看次数
2 回答
2
已接受的回答

Hi alex88,

to receive messages from your application you can use the rules engine to route messages to your application to overcome the 100msg/s quota for a single connection. You could route messages for example to Kinesis or SQS and then to your application.

How do you send messages to your devices? Do you send a message per device? You could think about using a single topic for your devices then you need to send only one message to address them all. Please see also the documentation for MQTT communication patterns.

In case you need to send a particular message to every device you could think about multiple MQTT connections or send publish requests in parallel.

Cheers,
Philipp

AWS
专家
已回答 1 年前
profile picture
专家
已审核 1 个月前
  • Hi Philipp, regarding sending we're good now, using rules engine was the only way. We send responses to every request we receive so per device answers by using a per device topic root. When you say publish requests which API you are referring to?

  • Hi alex88, I meant this Publish API: https://docs.aws.amazon.com/iot/latest/apireference/API_iotdata_Publish.html. You can also use an MQTT connection but as you mentioned you have to work around the 100msg/s quota with multiple connection. The idea is to use the Publish API with multiple requests in parallel.

1

While HTTPS actions works to overcome the 100 msg/s limit, it is not the optimal solution unless your backend application runs outside of AWS. A better option, as Philipp mentioned, is to use an SQS or a Kinesis action. Kinesis is recommended if message ordering is significant for your processing, otherwise you should use SQS.

With HTTPS action you need to consider how to handle the following scenarios:

  1. traffic peaks that your backend is not able to process will result in lost messages
  2. if your backend is for some reason unavailable, messages will be lost

Using SQS, you decouple the messages from the application consuming it. For example, you could write your application code using Lambda functions, that would automatically scale to consume the messages in the queue. In case your application has bugs and cannot process the messages, the messages would still be retained in the queue, thus creating a more reliable system.

As for publishing from the backend to AWS IoT Core, you SHOULD use https://docs.aws.amazon.com/iot/latest/apireference/API_iotdata_Publish.html. Your caller need to have credentials with permissions to invoke this API which is best achieved by using an IAM Role. If you use Lambda functions that would be the function execution role.

You can check a reference architecture showing these principles here:

https://aws.amazon.com/solutions/implementations/smart-product-solution/

AWS
专家
已回答 1 年前
  • Got it thank you, I'll switch to the Publish API

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则