Process ftp files with sensor readings

0

For an IoT project I need to ingest data (files with sensor data) coming from an (on-prem) FTP server. I have a Lambda function with a Cloudwatch cronjob that on an interval basis get the content of these files. I would like (a cold path) storing the raw data to DynamoDB and/or an S3 bucket and (a warm path) by using AWS IoT Analytics.

My question:

  •    Is it best to process the contents of the files and interface directly with DynamoDB and/or S3 and IoT Analytics (BatchPutMessage) or  
    
  •    Process the incoming files sensor readings publish them first to AWS IoT, and from there use rules for the cold and warm path?  
    

Thanks.
Guy

GuyD
질문됨 5년 전373회 조회
2개 답변
0

Of the two options you listed, I recommend the first one. You should just interface directly with S3 (or DynamoDB) and IoT Analytics if your data is not already being sent to the AWS IoT Core Message Broker.

You could alternatively use IoT Analytics for both the cold path and the warm path. You can use an IoT Analytics Datastore for cold storage in your S3 Bucket. [0] [1] You could then enrich the data in your IoT Analytics Dataset's query. This option is preferable if a SQL query is sufficient for your data enrichment needs.

Note that in both options I listed, you would ingest data into IoT Analytics through the BatchPutMessage API. What sets the second option apart is that you would not need to upload the data to S3 yourself; IoT Analytics would handle uploading the data to your bucket.

[0] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-iotanalytics-datastore-customermanageds3.html
[1] https://docs.aws.amazon.com/iotanalytics/latest/userguide/getting-started.html#aws-iot-analytics-step-create-datastore

Edited by: DavidWataws on Feb 27, 2020 7:52 PM

Edited by: DavidWataws on Feb 27, 2020 7:53 PM

답변함 5년 전
0

Many thanks.

GuyD
답변함 5년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠