Can a specific partition key be enforced for the Kinesis PutRecord action, via IAM policy or other means?

0

Given a producer sending records to a stream by calling the PutRecord action, is there any way to enforce that the producer is calling PutRecord with a specific partition key?

I've checked this documentation page Actions, resources, and condition keys for Amazon Kinesis, and it does not seem like a condition exists to allow an IAM policy to restrict the partition key.

1 Answer
1

Hello,

I understand that you have a query regarding enforcing a specific partition key for the Kinesis PutRecord API.

  • Firstly I would like to mention that currently the IAM action - PutRecord doesn't support any condition keys. Also, it only support resource restriction in terms of stream type -> arn:${Partition}:kinesis:${Region}:${Account}:stream/${StreamName}

  • Secondly, please note that partition key is a mandatory parameter when using PutRecord/PutRecords API call [1].

 PartitionKey

    Determines which shard in the stream the data record is assigned to. Partition keys are Unicode strings with a maximum length limit of 256 characters for each key. Amazon Kinesis Data Streams uses the partition key as input to a hash function that maps the partition key and associated data to a specific shard. Specifically, an MD5 hash function is used to map partition keys to 128-bit integer values and to map associated data records to shards. As a result of this hashing mechanism, all data records with the same partition key map to the same shard within the stream.

    Type: String

    Length Constraints: Minimum length of 1. Maximum length of 256.

    Required: Yes

  • As there is currently no way to restrict it from API's end using condition keys, you will have to create a custom implementation in your own application code to enforce the usage of a specific/single partition key restriction when calling the PutRecord API, however please note that the records will always land to a single shard in such a use case.

  • Finally, I also wanted to mention that the best practice provided by service team in regards to the partition key, is to use random partition keys such as uuid.


References:

[1] https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html

profile pictureAWS
SUPPORT ENGINEER
Yash_C
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions