Difficulty Implementing Basic Ingest Feature in AWS IoT Core

Question

I'm encountering difficulties while attempting to utilize the basic ingest feature within AWS IoT Core. My goal is to ingest data from a device into AWS IoT Core using the basic ingest feature, but I'm encountering errors or unexpected behavior.

Here's my code I've written to send the data to my AWS IoT Core :-

```
    import AWSIoTPythonSDK.MQTTLib as AWSIoT
    import random
    
    myClient = AWSIoT.AWSIoTMQTTClient("basic-ingest-thing")
    myClient.configureEndpoint("XXXXXXXXXXX-ats.iot.us-east-1.amazonaws.com",8883)
    myClient.configureCredentials("certs/AmazonRootCA1.pem","certs/private.pem.key","certs/certificate.pem.crt")
    myClient.connect()
    
    num = random.random()
    payload = "{{\"status\":\"{}\"}}".format(num)
    
    print(payload)
    
    myClient.publish("$aws/rules/basic_ingest_rule/data/wind/adodar", payload, 0)

```

Here is my AWS IoT Core Rules SQL Query :-

```
    SELECT *,timestamp() as ts FROM "$aws/rules/basic_ingest_rule/data/wind/adodar"

```

I'm sending the data to cloudwatch logs from AWS IoT Core rules

Here is my AWS IoT Core Policy :-

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iot:Connect",
      "Resource": "arn:aws:iot:us-east-1:XXXXXXXXXXX:client/*"
    },
    {
      "Effect": "Allow",
      "Action": "iot:Publish",
      "Resource": "arn:aws:iot:us-east-1:XXXXXXXXXXXX:topic/$aws/rules/basic_ingest_rule/*"
    }
  ]
}
```

Despite following these steps, I'm encountering issues such as:

1. **Data Not Being Received**: The data published by the device is not being received by AWS IoT Core, or it's not being routed as expected by the basic ingest rule.
2. **Troubleshooting Steps**: I've attempted to troubleshoot the problem by checking logs, verifying device connectivity, and reviewing AWS IoT Core documentation, but I haven't been able to identify the root cause of the issue.

I'm seeking guidance on how to troubleshoot this issue effectively or insights into common pitfalls when setting up and using the basic ingest feature in AWS IoT Core. Any assistance or suggestions would be greatly appreciated. Thank you!

Answer

Using [this page](https://docs.aws.amazon.com/iot/latest/developerguide/iot-basic-ingest.html) for the comments below.

Overall, you are close. The IoT Policy has the correct actions and resources to publish to the `$aws/rules` reserved topic used for basic ingest, and the python code looks correct too.

The one change would be in the rule itself. While your client publishes to `$aws/rules/basic_ingest_rule/data/wind/adodar` there are two things needed for the rule to be used for basic ingest. First, the rule must be name `basic_ingest_rule` in this instance, as that is how AWS IoT Core knows which rule to invoke for the incoming message.

Second, since the rule itself can be used for both basic ingest _or_ regular messages published to `data/wind/adodar`, the SQL syntax should look like this:

```sql
SELECT *,timestamp() as ts FROM '$data/wind/adodar'
```
The reason for removing the `$aws/...` portion in the `FROM` clause is that (from the doc link above):
> Your devices **and rules** can't subscribe to Basic Ingest reserved topics. For more information, see [Reserved topics](https://docs.aws.amazon.com/iot/latest/developerguide/reserved-topics.html#reserved-topics-rule).

Emphasis mine. Also, if the rule will only be used for basic ingest, the `FROM` clause is optional. More details can be found in [Using Basic Ingest](https://docs.aws.amazon.com/iot/latest/developerguide/iot-basic-ingest.html#iot-basic-ingest-use) section of the documentation.

Please let me know if this helps clear things up and more importantly, works. If you are still have problems, please post a comment.

Difficulty Implementing Basic Ingest Feature in AWS IoT Core

Relevant content