Optimizing AWS IoT Costs with Basic Ingest: Utilizing Ingest Topics and Publishing Messages

Here are a few suggestions to help configure AWS IoT Basic Ingest and publish messages to it:

Regarding the topic structure, the recommended format is correct - $aws/rules/<rule-name>/<rest-of-topic>. For your use case, the topic would be $aws/rules/routeToKinesis/Building1/room2/tubelight. I don't know if it was a typo, but do not forget the "rules" element before the rule name in your topic.

As and example, the SQL statement in the rule needs to use a topic() function to extract the building, room and device information from the rest of the topic:

SELECT
  topic(2) as building, 
  topic(3) as room,
  topic(4) as device
FROM
  '$aws/rules/routeToKinesis/Building1/room2/tubelight'

This extracts Building1, room2 and tubelight into separate fields that can be processed by the rule action. You can use any other function to send any part of the payload to downstream systems as usual.

To publish messages to IoT Basic Ingest topics, you can use one of the AWS IoT Device SDKs. Here is some sample Python code to publish a message using the MQTT SDK:

import AWSIoTPythonSDK.MQTTLib as AWSIoTPyMQTT

myAWSIoTMQTTClient = AWSIoTPyMQTT.AWSIoTMQTTClient("basicIngestPublisher")
myAWSIoTMQTTClient.configureEnd[]()point("xxxxxxxxxxxxx.iot.us-east-1.amazonaws.com", 8883)
myAWSIoTMQTTClient.configureCredentials("../../../certs/AmazonRootCA1.pem", "../../../certs/private.pem.key", "../../../certs/certificate.pem.crt")

myAWSIoTMQTTClient.connect()
myAWSIoTMQTTClient.publish("$aws/routeToKinesis/Building1/room2/tubelight", "{\"status\":\"on\"}", 0)

This publishes a simple JSON message to the desired topic.

Overall, when using IoT Basic Ingest:

• Follow AWS best practices like the Well-Architected Framework pillars to ensure security, reliability, performance, and operational excellence.

• Monitor metrics like publish success rate to identify any issues.

• Use IoT Device Defender to secure devices publishing to IoT Basic Ingest.

• Use IoT Device Management to easily manage fleet of devices.

Resources:

• Reducing messaging costs with Basic Ingest
• AWS IoT SQL reference
• AWS IoT Device SDKs, Mobile SDKs, and AWS IoT Device Client
• AWS Well-Architected Frameworks - IoT Lens
• Monitoring AWS IoT
• AWS IoT Device Defender
• AWS IoT Device Management

Let me know if you have any other questions!

A few issues:

1. Use of MQTT wildcard characters in policies

See https://docs.aws.amazon.com/iot/latest/developerguide/pub-sub-policy.html#pub-sub-topic-wildcard for guidance. Basically you can only use MQTT wildcards for subscribing to topic filters, so your policy for publishing should include the IAM wildcard * instead of #

2. In the first answer provided, there is an error in the client code example. You must publish to $aws/rules/<rulename>/<topic> - the author omitted the rules portion.

3. In the first answer provided, the SQL statement needs improvement. It should read as below:

SELECT
  * as message,
  topic(1) as building, 
  topic(2) as room,
  topic(3) as device
FROM
  '$aws/rules/routeToKinesis/+/+/+'

Note I have include * as message, which will populate the received event with the message content. If you omit this, you'd only receive the topic metadata (building, room, device) in the received event.

E.g. if the following message is sent to $aws/rules/routeToKinesis/Building1/room2/tubelight

{ "foo": "bar" }

the event sent to the rule destination will be formatted as follows:

{
  "building": "Building1",
  "room": "room2",
  "device": "tubelight",
  "message": { "foo": "bar" }
}