Table Size growth and cost implication of using DynamoDb table for kinesis Stream Checkpointing

0

We are trying to use consume Kinesis Stream from Nifi and that requires us to use DynamoDb Table for Checkpointing at the source account. So a few question related to this effort.

  1. Can we use the DynamoDB table on the Consumer account instead of the Source Account ?
  2. How does this table grow eventually and what would be cost implication of the growth?
gefragt vor einem Jahr274 Aufrufe
1 Antwort
0

To use a DynamoDB table for checkpointing at the consumer account you will need to:

  • Create a new DynamoDB table in the consumer account that has the same structure as the checkpointing table in the source account.
  • Update the kinesis consumer code in Nifi to use the new DynamoDB table in the consumer account
  • Test the new configuration to ensure that checkpointing is working

Please make sure the correct permissions are updated to allow cross account access to the new DynamoDB. Please check with the Nifi documentation to get the exact permissions needed.

In response to the table growth, the database should remain stable so long as your shard count doesn't change. When shards rollover periodically, this number is briefly elevated, but if your application is processing at par with the throughput, it would go back down pretty fast.

AWS
SUPPORT-TECHNIKER
Luke_D
beantwortet vor einem Jahr

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen