IoT Rule Not Matching Published Messages

Question

Objective:
We are trying to trigger an AWS Lambda function (IoTMessageParser) when MQTT messages are published to the topic esp8266/data from an ESP8266 device. The Lambda should transform the data and send it to Kinesis Data Firehose (PUT-S3-IrXpM), which then delivers to S3 bucket.

Current State:

✅ ESP8266 device successfully publishes to topic esp8266/data every 60 seconds
✅ Messages arrive at AWS IoT Core (confirmed in CloudWatch IoT logs - Publish-In events appear)
✅ Lambda function works correctly when invoked directly (manual test succeeds, writes to Firehose)
✅ All permissions configured (Lambda has Firehose access, Firehose has S3 access, IoT has Lambda invoke permissions)
✅ S3 bucket policy allows Firehose to write
❌ IoT Rules do NOT match the incoming messages - no RuleMatch or RuleExecution events in CloudWatch logs

Troubleshooting Steps Attempted:

Created multiple IoT Rules (SendToLambda, TestRule, FreshRule) with identical SQL statements - none match messages
Verified SQL syntax - tried with and without quotes: SELECT * FROM esp8266/data and SELECT * FROM 'esp8266/data'
Tried wildcard pattern - SELECT * FROM 'esp8266/#' - still no match
Deleted and recreated rules - fresh rule creation doesn't resolve issue
Verified Lambda permissions - resource-based policies automatically created for each rule showing proper ARNs and conditions
Confirmed messages arrive - CloudWatch IoT logs show Publish-In events with correct topic name esp8266/data
Tested Lambda directly - function executes successfully and writes to Firehose when manually invoked
Enabled/re-enabled CloudWatch logging - DEBUG level logging active
Deactivated conflicting rules - ensured only one rule active at a time
Verified rule status - confirmed Active status with green checkmark

CloudWatch Log Example (Current Behavior):

No RuleMatch or RuleExecution events follow this Publish-In event.

Questions:
Why are IoT Rules not matching messages published to esp8266/data when CloudWatch logs confirm the messages are arriving at IoT Core? Is there a cached configuration issue, service problem in us-east-1, or account-level restriction preventing rule evaluation?

Expected Behavior:
After a Publish-In event for topic esp8266/data, we should see:

RuleMatch event for FreshRule
RuleExecution event showing Lambda action triggered
Lambda invocation count increasing
Data flowing to Firehose and S3
Actual Behavior:
Only Publish-In events appear. No rule evaluation occurs despite active rule with matching SQL statement.

Thanks

re:Post Agent · Answer

Based on the information provided, there are several potential reasons why your IoT rules are not matching the published messages:

1. **SQL Statement Format**: The SQL statement in your IoT rule might need adjustment. For the topic `esp8266/data`, the correct format should be:
```
SELECT * FROM 'esp8266/data'
```
Note that the topic name should be enclosed in single quotes.

2. **External Service Capacity**: Before rule execution, ensure that the external services linked to your rule (Lambda and Kinesis Data Firehose) have sufficient throughput and capacity units for your application. If these services are throttling or experiencing issues, it could prevent rule execution.

3. **WHERE Clause Filtering**: If you've added any WHERE clauses to your rule, they might be filtering out your messages. Try using a simple rule without any WHERE conditions first to confirm basic functionality.

4. **Rule Action Configuration**: Verify that the rule action is correctly configured with the proper Lambda function ARN and IAM role permissions.

5. **Message Format**: The message payload format might not match what the rule is expecting. Ensure your ESP8266 is publishing valid JSON if your rule expects JSON data.

To further troubleshoot:

- Try creating a simpler rule that just republishes to another MQTT topic (using the republish action) to see if basic rule matching works
- Check if there are any errors in the CloudWatch Logs related to rule execution failures
- Verify that the IAM role associated with your rule has the necessary permissions to invoke your Lambda function
- Ensure that your Lambda function is in the same region as your IoT Core setup

If all these checks pass and you're still experiencing issues, it might be worth testing with the AWS IoT MQTT test client to publish test messages and see if they trigger your rules.  
**Sources**  
[Harnessing the power of AWS IoT rules with substitution templates | The Internet of Things on AWS – Official Blog](https://aws.amazon.com/blogs/iot/unleashing-the-power-of-aws-iot-rules-substitution-templates-in-actions/)  
[Diagnosing rules issues - AWS IoT Core](https://docs.aws.amazon.com/iot/latest/developerguide/diagnosing-rules.html)  
[Best practices for using MQTT topics in the AWS IoT Rules Engine - Designing MQTT Topics for AWS IoT Core](https://docs.aws.amazon.com/whitepapers/latest/designing-mqtt-topics-aws-iot-core/best-practices-for-using-mqtt-topics-in-the-aws-iot-rules-engine.html)

Anne · Answer

Not sure, if you've figured it out already. The only difference I see in my setup is that I actually used double-quotes for the topic name in the SQL of the rule.

IoT Rule Not Matching Published Messages

Add your answer

Relevant content