AWS IoT Analytics Datasets returning column names but no rows

Question

Hi there,
I have setup an ETL pipeline in AWS IoT Analytics (channels, pipeline, data store) and now want to create IoT Analytics **Datasets**. When creating the dataset, the query does not return any data, except for the column names. For testing purposes, I am using the default select query: `SELECT * FROM my_data_store LIMIT 10`.
The datastore contains data, e.g., I ran a Glue Crawler over the data in the data store, queried it with Athena and got correct results. I also manually downloaded the data from S3, and the data can be queried locally. But the Datasets CSV keeps being empty (except for all the column names). The Datasets content view in the AWS Console also only displays the Column Names.
To be on the safe side, I further added full S3 permissions to my IAM Roles, but without any success.

What is the problem here? Is the *dataset* functionality working for any of you?

Best,
Jon

Accepted Answer

Are you using your own S3 bucket for storing/managing the data then you must grant AWS IoT Analytics permission to perform these actions on your Amazon S3 bucket for you. If you use the data store as a source for an SQL query dataset, set up an Amazon S3 bucket policy that grants AWS IoT Analytics permission to invoke Amazon Athena queries on the contents of your bucket. Please refer to the https://docs.aws.amazon.com/iotanalytics/latest/userguide/policy.html link for more details.

If you update the options or permissions of your customer managed data store, you might need to reprocess channel data to ensure that any previously ingested data is included in dataset contents. For more information, see https://docs.aws.amazon.com/iotanalytics/latest/userguide/reprocessing.html#aws-iot-analytics-reprocessing.

* s3:GetBucketLocation
* s3:PutObject
* s3:DeleteObject

AWS IoT Analytics Datasets returning column names but no rows

Relevant content