Questions tagged with Amazon Data Firehose
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
Does PutRecord SDK has retry feature as KPL? I have found the below link about handling failures
https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-sdk.html
Can we say...
1
answers
0
votes
910
views
asked 4 months agolg...
Hi,
We have a scenario where we want to send a JSON object that contains a nested list of records into Kinesis Firehose from an IoT Core rule, and have this list split into multiple records, with...
1
answers
0
votes
901
views
asked 4 months agolg...
I'm designing a system that will listen to over 1000 streams (1 for each device), and should be able to process the streamed data to take action when needed. I need the processing and taking action to...
0
answers
0
votes
1413
views
asked 5 months agolg...
Hello, dear community!
I have configured AWS Kinesis Data Firehose to stream data to an HTTP endpoint, with an S3 backup configuration enabled for storing all data in an S3 bucket. Despite this setup,...
0
answers
0
votes
198
views
asked 5 months agolg...
I need to call PutSubscriptionFilter for a Firehose data stream. I created my role that trusts the logs.amazonaws.com service and allows it to put records. There is a permission boundary on my user...
0
answers
0
votes
117
views
asked 5 months agolg...
How to correctly estimate the expected JQ Processing Hours for dynamic partitioning in Kinesis Firehose as being asked in pricing calculator ? It's not very clear from the documentation. Does every...
1
answers
0
votes
241
views
asked 6 months agolg...
pipeLine : In order to index data to opensearch, kinesis is called from lambda and the data is forwarded to opensearch.
when sending demo data to 'Opensearch' from 'kdf', 'Opensearch' set index...
1
answers
0
votes
575
views
asked 6 months agolg...
I have a glue table specified that refers to a location in S3. I noticed that when Firehose puts data there in parquet format it might be not always compressed.
Will it cause issues for Glue table,...
1
answers
0
votes
297
views
asked 6 months agolg...
I have messages arriving into S3 from Firehose - the schema of messages is JSON and has two structs "header" and "body", each of which contains simple data types. Glue table so far has partitions that...
1
answers
0
votes
350
views
asked 6 months agolg...
1. Firehose has the following configuration:
![Enter image description here](/media/postImages/original/IM0gtmftmZSSqBsYC9pSOCTg)
2. Glue table doesn't specify that source is encrypted.
3. The...
0
answers
0
votes
116
views
asked 7 months agolg...
1. I am trying to understand how to estimate the end result of pricing for Glue crawlers for data being ingested by AWS Kinesis Firehose service with the max possible buffer setting while glue...
0
answers
0
votes
173
views
asked 7 months agolg...
I have a Glue table created from schema in Glue schema registry that is in JSON draft 07 format.
I tried setting up a Firehose that converts incoming JSON data into Parquet as it saves it into S3,...
0
answers
0
votes
198
views
asked 7 months agolg...