cannot create IOT rule action to MSK cluster


When configuring an IOT rule action for MSK, I get the following error.

1 validation error detected: Value '[]' at 'topicRulePayload.actions.1.member.kafka.headers' failed to satisfy constraint: Member must have length greater than or equal to 1

I have followed the instruction for configuration from these guides. I can successfully produce and consume MSK messages via Cloud9 IDE.

Field Notes: Deliver Messages Using an IoT Rule Action to Amazon Managed Streaming for Apache Kafka

How to integrate AWS IoT Core with Amazon MSK

I do not understand how this error relates to the configuration. There is no option to configure headers in the rule action UI.

1 Answer

I ran into the same issue today, seems like a bug with the AWS Console because I was able to create the rule through the CLI.

Here is the command:

aws iot create-topic-rule --rule-name <RULE_NAME> --topic-rule-payload file://iot.json

and the input file:

    "sql": "<SQL_COMMAND>",
    "description": "",
    "actions": [
            "kafka": {
                "destinationArn": "arn:aws:iot:******************",
                "topic": "<TOPIC>",
                "clientProperties": {
                    "acks": "1",
                    "bootstrap.servers": "****************************",
                    "compression.type": "none",
                    "key.serializer": "org.apache.kafka.common.serialization.StringSerializer",
                    "sasl.mechanism": "SCRAM-SHA-512",
                    "sasl.scram.password": "************",
                    "sasl.scram.username": "*******************",
                    "security.protocol": "SASL_SSL",
                    "value.serializer": "org.apache.kafka.common.serialization.ByteBufferSerializer"
answered 9 months ago
  • Thank you for your answer! I had a feeling it was a bug in AWS Console, but being a noob here, I wasn't sure. I tried the CLI approach and that worked for me. kafka processing messages!

    I'm not accepting this as the answer, because I believe the answer is for AWS to fix the Console.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions