Having trouble getting the Java Example Consumer to work / understanding Kinesis Data Streams concepts

0

So far, I read this article: https://docs.aws.amazon.com/streams/latest/dev/key-concepts.html?icmpid=docs_console_unmapped

Now I am trying to implement a minimalistic example according to this article

https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/examples-kinesis-stream.html

The following things are not clear to me:

What resource do I have to specify in the consumerARN? The Java Application is a Spring Boot Application running on AWS Fargate. So will the ARN be the ECS Task? How do I make the shardId deterministic? I have no control over the client that is putting the records on the data stream. Polling the listShards method continuously does not seem like a good idea. I also don't know how many records will be put on the stream in a given timeframe.

What exactly is the relation between the partitionKey and the shardId? Can different partitionKeys result in the same shardId? If the client would always use the same partitionKey for all records, will the shardId stay the same forever?

If there were multiple shards, should I handle each shard with a separate SubscribeToShardRequest on a new Thread? If yes, how many Threads should I create at max on an ECS instance with 2GB RAM and 1vCPU / how many shards can a a Kinesis Data Stream have at maximum?

已提問 2 年前檢視次數 106 次
沒有答案

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南