Unanswered Questions tagged with Amazon Data Firehose
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I'm trying to configure a streaming pipeline using Amazon Firehose with the new Snowflake integration. Both the AWS and Snowflake accounts are in **us-east-1** region.
I have an existing integration...
0
answers
0
votes
99
views
asked 2 months agolg...
I'm designing a system that will listen to over 1000 streams (1 for each device), and should be able to process the streamed data to take action when needed. I need the processing and taking action to...
0
answers
0
votes
1116
views
asked 4 months agolg...
Hello, dear community!
I have configured AWS Kinesis Data Firehose to stream data to an HTTP endpoint, with an S3 backup configuration enabled for storing all data in an S3 bucket. Despite this setup,...
0
answers
0
votes
190
views
asked 4 months agolg...
I need to call PutSubscriptionFilter for a Firehose data stream. I created my role that trusts the logs.amazonaws.com service and allows it to put records. There is a permission boundary on my user...
0
answers
0
votes
111
views
asked 5 months agolg...
1. Firehose has the following configuration:
![Enter image description here](/media/postImages/original/IM0gtmftmZSSqBsYC9pSOCTg)
2. Glue table doesn't specify that source is encrypted.
3. The...
0
answers
0
votes
110
views
asked 6 months agolg...
1. I am trying to understand how to estimate the end result of pricing for Glue crawlers for data being ingested by AWS Kinesis Firehose service with the max possible buffer setting while glue...
0
answers
0
votes
168
views
asked 6 months agolg...
I have a Glue table created from schema in Glue schema registry that is in JSON draft 07 format.
I tried setting up a Firehose that converts incoming JSON data into Parquet as it saves it into S3,...
0
answers
0
votes
189
views
asked 6 months agolg...
We want to enable the schema validation for Firehose streams as described here: https://docs.aws.amazon.com/firehose/latest/dev/create-transform.html
Quote:
"To enable schema validation, check the...
0
answers
1
votes
167
views
asked 7 months agolg...
Hi! I'm copying this tutorial almost exactly: https://aws.amazon.com/blogs/big-data/serverless-logging-with-amazon-opensearch-serverless-and-amazon-kinesis-data-firehose/
I've set up OpenSearch...
0
answers
0
votes
159
views
asked 7 months agolg...
* We are running an opentelemetry collector that exports the OTEL traces to Kinesis
* A Kinesis Firehose Stream transforms the records with a lambda and saves them to the OpenSearch Service...
0
answers
0
votes
198
views
asked 9 months agolg...
How do you debug your Eventbridge pipe line with self hosted kafka as source and firehose as targetlg...
Hello,
I am having hard time getting the eventbridge pipeline to work. I tried all different combinations. I am using the self hosted kafka cluster(cloudkarafka.com deployed on aws) and my target is...
0
answers
0
votes
104
views
asked a year agolg...
I am trying to confirm my OpenSearch setup is working but I am not able to insert new data to confirm this. Here are a list of commands and response I am seeing from OpenSearch. My goal is to hook...
0
answers
0
votes
140
views
asked a year agolg...