- Newest
- Most votes
- Most comments
You are correct that Amazon Kinesis Data Analytics typically consumes data from Kinesis Data Streams or Kinesis Data Firehose, not directly from S3. However, the question and the correct answer (D) align with a valid AWS architecture, albeit with some implied steps.
In this scenario, Kinesis Data Firehose can be configured to convert incoming JSON data to Apache Parquet format before storing it in S3. Firehose can also handle the encryption requirement. While Kinesis Data Analytics doesn't directly read from S3, it can analyze the data as it flows through Firehose before being stored in S3.
The question asks for a solution with the least operational overhead, and option D provides that by using fully managed services (Firehose and Kinesis Data Analytics) without the need for additional components like Lambda functions or EMR clusters.
You're right that for analyzing data already in S3, especially in Parquet format, Amazon Athena would be a more typical choice. However, the question emphasizes near-real-time analysis, which is a strength of Kinesis Data Analytics.
In summary, while your understanding of the typical use cases is correct, the proposed solution (D) can work within the constraints of the question, focusing on real-time analysis and minimal operational overhead.
Sources
Data ingestion methods - Storage Best Practices for Data and Analytics Applications
What Is Amazon Kinesis Data Analytics for SQL Applications? - Amazon Kinesis Data Analytics for SQL Applications Developer Guide
Amazon Kinesis - Big Data Analytics Options on AWS
You're absolutely right in questioning the correctness of option D in the context of analyzing Parquet data from S3. Let's break this down logically:
- Amazon Kinesis Data Analytics (KDA)
KDA is designed to consume streaming data directly from Kinesis Data Streams (KDS) or Kinesis Data Firehose (KDF).
KDA does NOT read data from S3. It operates on streaming data, not batch data stored in Parquet format.
If data is already in S3, the best service for querying would be Amazon Athena or AWS Glue, not KDA.
- Analyzing Parquet Data from S3
If the company stores data in Apache Parquet format in S3, the best services for analysis would be:
Amazon Athena – Serverless querying of Parquet data in S3.
Amazon EMR – If batch processing is required using Apache Spark/Hive.
AWS Glue – If performing ETL transformations before querying.
- Evaluating the Answer Choices
Option A & B use Kinesis Data Streams to store data in S3. But Kinesis Data Streams itself does not store data in S3, only processes streaming data.
Option C & D use Kinesis Data Firehose to store data in S3, which is correct. However:
Option C: Uses EMR for analysis, which is a valid option.
Option D: Uses Kinesis Data Analytics for analysis, which is wrong because KDA cannot read from S3.
Final Conclusion
Your understanding is correct – Amazon Kinesis Data Analytics cannot analyze Parquet data from S3.
The exam answer D is incorrect because KDA cannot directly analyze S3 data.
The most accurate solution would likely involve Kinesis Data Firehose (for ingesting real-time data into S3) + Athena or EMR (for analyzing Parquet data from S3).
You're thinking in the right direction!
answered 2 years ago
Relevant content
asked 3 years ago
