Errors at dimensions (empty value) in Timestream from an IoT Rule

Question

Hello:
I'm trying to insert data into Timestream from AWS IoT, where I created a rule as:

```
SELECT * FROM 'dataTopic'
ACTIONS: 
Write a message into a Timestream 
     DB: test, 
     table:  sensors, 
     dimension name: device, 
     dimension value: ${device}
Republish a message to an AWS IoT topic: test
ERROR ACTION:
Republish a message to an AWS IoT topic: error
```

And publishing data as:

```
{
       "device": "abc123",
       "temperature": "24.50",
       "humidity": "49"
}

```
**works fine.**
**NOW, my real data** is actually like this;

```
{
  "state": {
    "reported": {
       "device": "abc123",
       "temperature": "24.50",
       "humidity": "49"
   }
  }
}
```

So I had to modify my Rule as:
`SELECT state.reported.* FROM 'dataTopic'`
but when I test it, I get an error from Timestream as it seems
```
 "failures" : [ {
    "failedAction" : "TimestreamAction",
    "failedResource" : "test#sensors",
    "errorMessage" : "Failed to write records to Timestream. The error received was 'Errors at dimensions.0: [Dimension value can not be empty.]'. Message arrived on dataTopic, Action: timestream, Database: test, Table: sensors"
```

However, **checking the data received at topic test, I don't see differences** with the original data;

```
{
  "device" : "abc123",
  "temperature" : "24.50",
  "humidity" : "49"
}
```
What could be the problem? So far, I see same data being ingested but for some reason, Timestream is seeing something different. I tried to use Cloudwatch to see what exactly is Timestream receiving, but I couldn't see the logs from this service.
I would appreciate any help.
Thanks

Accepted Answer

The root cause for the issue is that you can't use the output of IoT SQL statements in substitution templates. So you can use `SELECT state.reported.*` to generate an input for the Timestream action, but for dimension value you can only reference the device by ${state.reported.device}.

If for whatever reason you can't use  ${state.reported.device} and need to use ${device} for dimension, the simplest way to work around here is to use "Republish" action with the content of `SELECT state.reported.*` to another IoT Rule. To make it cost efficient you can republish to another IoT Rule using basic Ingest, e.g. using topic like `$aws/rules/MyRule`. However, for your use case I am quite confident that the approach I describe above should work and you don' need a second IoT Rule.

Errors at dimensions (empty value) in Timestream from an IoT Rule

Relevant content