Questions tagged with Amazon Kinesis
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
how to you mange application state when using KCL.
I create a simple application which will save my application state on efs when creating checkpoint, but the state is per shard. it willnot work when...
0
answers
0
votes
104
views
asked 2 years agolg...
How does data flow as you scale up Kinesis Data Steams and add KPU's to Kinesis Data Analytics.lg...
I have an event processing architecture that's basically KPL Producer -> Kinesis Data Streams(Raw) -> KDA(Flink Aggregation) -> Kinesis Data Streams(Agg) -> Lambda -> Timestream . With a basic setup...
1
answers
0
votes
289
views
asked 2 years agolg...
Error:
HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas. The types are incompatible and cannot be coerced. The column 'isfallbackkey' in table...
0
answers
0
votes
205
views
asked 2 years agolg...
My one AWS hosted application need to send PDF or DOC or JPG any format files to http, SQS , https type subscriptions through SNS. Is it possible? Can you please provide any AWS reference tutorial...
2
answers
0
votes
5242
views
asked 2 years agolg...
In the documentation (https://docs.aws.amazon.com/firehose/latest/dev/controlling-access.html ), there are 2 set of IPs for us to whitelist when using Splunk or RedShift as Kinesis Data Firehose...
0
answers
0
votes
114
views
asked 2 years agolg...
I am using KCL 1.14.8 for Java. My Kinesis only have one shard, and I put a record into Kinesis every 30 seconds. For each record, the RecordProcessor will process it, processing each record may take...
0
answers
0
votes
126
views
asked 2 years agolg...
I am hoping to create a stream of messages, and the stream will be defined by providing an S3 bucket + prefix, and will stream:
1. a list of all existing objects at the given prefix
2. new objects...
0
answers
0
votes
143
views
asked 2 years agolg...
I am consuming Amazon Connect CTRs through Amazon Kinesis and inserting my data into Postgres. I am facing very unexpected behavior from Kinesis and Lambda function. Whenever a CTR record comes...
2
answers
0
votes
312
views
asked 2 years agolg...
Hi,
I am working on a system that uses kinesis to ingest data, this data is then processed by a lambda function and kinesis delivery stream. We have CloudWatch alarms set up that trigger if the...
1
answers
0
votes
2395
views
asked 2 years agolg...
When hooking up a lambda to a kinesis stream, the event source can choose a starting position. Settings like TRIM_HORIZON and timestamps allow to replay data that is already in the stream. When using...
1
answers
0
votes
1668
views
asked 2 years agolg...
Hi.
I have followed the instructions to the letter in the YouTube video that Paul Shiner did, and am getting the error:
[INFO ] [24-07-2022 18:22:56:030.560 GMT] Using region: eu-west-2
[INFO ]...
2
answers
1
votes
353
views
asked 2 years agolg...
When using Kinesis firehose to opensearch / elasticsearch, while the delivery stream is super convenient, one major limitation I find is that one cannot override the timestamp field that is use to...
0
answers
0
votes
123
views
asked 2 years agolg...