Questions tagged with Amazon Kinesis Data Streams
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I have a Kinesis stream that is persisting (inserting) data to an iceberg table, via a Glue streaming job. I'm following the glue streaming pattern as published...
2
answers
0
votes
1623
views
asked 4 months agolg...
Hello,
I am getting below exception when two glue jobs consumes same stream. The Stream has multiple shards.
com.amazonaws.services.kinesis.model.ProvisionedThroughputExceededException: Rate exceeded...
1
answers
0
votes
1531
views
asked 4 months agolg...
Hi,
Does PutRecord SDK has retry feature as KPL? I have found the below link about handling failures
https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-sdk.html
Can we say...
1
answers
0
votes
1340
views
asked 5 months agolg...
Hi there,
I am trying to establish ETL processing from our existing AWS Kinesis data stream to snowflake. We are currently using KCL which is extremely difficult to maintain as we do not have the...
1
answers
0
votes
1696
views
asked 5 months agolg...
[DestinationConfig](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateEventSourceMapping.html#SSS-CreateEventSourceMapping-request-DestinationConfig) is a configuration object that specifies the...
1
answers
0
votes
1284
views
asked 5 months agolg...
I'm designing a system that will listen to over 1000 streams (1 for each device), and should be able to process the streamed data to take action when needed. I need the processing and taking action to...
0
answers
0
votes
1981
views
asked 6 months agolg...
AWS Lambda now supports Parallelization factor to process single ddb shard by multiple concurrent invocations....
2
answers
0
votes
528
views
asked 6 months agolg...
I'm working with a fleet of 300+ IoT devices sending telemetry data to a Kinesis Data Stream at a rate of 25 million messages per day. The data is stored in a MongoDB database using a schema with...
1
answers
0
votes
342
views
asked 6 months agolg...
0
Hello, I am running a glue streaming job where I read data from KDS, do some aggregation and filter it. I want to trigger SNS on this filtered DF and include its column values in the SNS message...
1
answers
0
votes
250
views
asked 7 months agolg...
I've created an external table on top of the Kinesis Data Streams and created a materialized view on top of the external table. It is running properly when I added a new column key/value JSON pair to...
1
answers
0
votes
381
views
asked 7 months agolg...
Importing the existing resources into CloudFormation management: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import.html, It doesn't seem to explain an option to import...
1
answers
0
votes
275
views
asked 8 months agolg...
Hi,
We have a system where we ingest data via kinesis data stream, connected to kinesis firehose, and outputted to s3. Occasionally and seemingly arbitrarily, our monitor on GetRecords iterator age...
1
answers
0
votes
303
views
asked 8 months agolg...