Process ftp files with sensor readings

0

For an IoT project I need to ingest data (files with sensor data) coming from an (on-prem) FTP server. I have a Lambda function with a Cloudwatch cronjob that on an interval basis get the content of these files. I would like (a cold path) storing the raw data to DynamoDB and/or an S3 bucket and (a warm path) by using AWS IoT Analytics.

My question:

  •    Is it best to process the contents of the files and interface directly with DynamoDB and/or S3 and IoT Analytics (BatchPutMessage) or  
    
  •    Process the incoming files sensor readings publish them first to AWS IoT, and from there use rules for the cold and warm path?  
    

Thanks.
Guy

GuyD
已提問 5 年前檢視次數 373 次
2 個答案
0

Of the two options you listed, I recommend the first one. You should just interface directly with S3 (or DynamoDB) and IoT Analytics if your data is not already being sent to the AWS IoT Core Message Broker.

You could alternatively use IoT Analytics for both the cold path and the warm path. You can use an IoT Analytics Datastore for cold storage in your S3 Bucket. [0] [1] You could then enrich the data in your IoT Analytics Dataset's query. This option is preferable if a SQL query is sufficient for your data enrichment needs.

Note that in both options I listed, you would ingest data into IoT Analytics through the BatchPutMessage API. What sets the second option apart is that you would not need to upload the data to S3 yourself; IoT Analytics would handle uploading the data to your bucket.

[0] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-iotanalytics-datastore-customermanageds3.html
[1] https://docs.aws.amazon.com/iotanalytics/latest/userguide/getting-started.html#aws-iot-analytics-step-create-datastore

Edited by: DavidWataws on Feb 27, 2020 7:52 PM

Edited by: DavidWataws on Feb 27, 2020 7:53 PM

已回答 5 年前
0

Many thanks.

GuyD
已回答 5 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南