Questions tagged with Amazon Data Firehose
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I have a glue table specified that refers to a location in S3. I noticed that when Firehose puts data there in parquet format it might be not always compressed.
Will it cause issues for Glue table,...
1
answers
0
votes
313
views
asked 7 months agolg...
I have messages arriving into S3 from Firehose - the schema of messages is JSON and has two structs "header" and "body", each of which contains simple data types. Glue table so far has partitions that...
1
answers
0
votes
369
views
asked 7 months agolg...
1. Firehose has the following configuration:
![Enter image description here](/media/postImages/original/IM0gtmftmZSSqBsYC9pSOCTg)
2. Glue table doesn't specify that source is encrypted.
3. The...
0
answers
0
votes
122
views
asked 7 months agolg...
1. I am trying to understand how to estimate the end result of pricing for Glue crawlers for data being ingested by AWS Kinesis Firehose service with the max possible buffer setting while glue...
0
answers
0
votes
178
views
asked 7 months agolg...
I have a Glue table created from schema in Glue schema registry that is in JSON draft 07 format.
I tried setting up a Firehose that converts incoming JSON data into Parquet as it saves it into S3,...
0
answers
0
votes
207
views
asked 7 months agolg...
What is the cloud watch metric for failed glue jobs/runs?
Simply put I need a metric for failed glue jobs/runs.
The GlueExceptionAnalysisListener seems to be the only thing capturing failed glue...
2
answers
0
votes
570
views
asked 7 months agolg...
Hi,
We have a system where we ingest data via kinesis data stream, connected to kinesis firehose, and outputted to s3. Occasionally and seemingly arbitrarily, our monitor on GetRecords iterator age...
1
answers
0
votes
292
views
asked 8 months agolg...
We want to enable the schema validation for Firehose streams as described here: https://docs.aws.amazon.com/firehose/latest/dev/create-transform.html
Quote:
"To enable schema validation, check the...
0
answers
1
votes
179
views
asked 8 months agolg...
Hello all,
I have a Kinesis Data Stream in account A, and I need to stream the events from it to Kinesis Firehose in account B.
From what I see, there is no direct integration (even using IAM roles...
2
answers
0
votes
611
views
asked 8 months agolg...
We have a Kinesis Data Firehose that have
| Buffer hints | value |
| --- | --- |
| Buffer size | 128 MiB |
| Buffer interval | 900 seconds |
| Retry | 300 seconds |
Data Freshness = Time in Buffer +...
1
answers
1
votes
392
views
asked 9 months agolg...
I'm trying to use IoT Events Alarm actions with custom payload and Firehose as action target. When i look into the documentation i can't find any example of the syntax required to write a custom...
1
answers
0
votes
281
views
asked 9 months agolg...
Hi! I'm copying this tutorial almost exactly: https://aws.amazon.com/blogs/big-data/serverless-logging-with-amazon-opensearch-serverless-and-amazon-kinesis-data-firehose/
I've set up OpenSearch...
0
answers
0
votes
179
views
asked 9 months agolg...