Amazon MSK Topic
0
Hi AWS,
Is it better and efficient to use multiple Kafka topics to deal with storing the request and response to process one at a time OR having one Kafka topic is good enough? What are the pros and cons of each approach?
Thanks
demandé il y a un an448 vues
1 réponse
- Le plus récent
- Le plus de votes
- La plupart des commentaires
0
What is your usecase? Normally depending on the data retention requirement the number of partitions in a cluster is a good a indicator.
Check the best practises here: https://docs.aws.amazon.com/msk/latest/developerguide/bestpractices.html
Also you can check this blog for MSK serverless: https://aws.amazon.com/blogs/big-data/create-more-partitions-and-retain-data-for-longer-in-your-msk-serverless-clusters/
répondu il y a un an
Contenus pertinents
- demandé il y a un an
- demandé il y a 7 mois
- demandé il y a 2 mois
AWS OFFICIELA mis à jour il y a 4 mois- AWS OFFICIELA mis à jour il y a 3 ans
- AWS OFFICIELA mis à jour il y a un an
Hi @rePost-User-2054795, here is the full use case:
Database of customer numbers ==>Adapter (any language) reading that 300K customer numbers once a month and posting one customer number at a time to a remote REST API and receives the response and storing the response in S3. How do we get this done efficiently considering the facts on throughput, data storage, latency, error handling, retry mechanism, data loss prevention, queues/topics, folder structure, cost and accomplish in few hours. I also want to use Amazon EKS in the same architecture. Can you help me out ?