iot route to kafka-built-by-myself vpc failed,but is passed before few days

0
"failures": [
        {
            "failedAction": "KafkaAction",
            "failedResource": "fromIot",
            "errorMessage": "KafkaAction failed to send a message to the specified bootstrap servers. Topic fromIot not present in metadata after 1000 ms.. Message arrived on: $aws/events/presence/disconnected/17ce7c11b83b4, Action: kafka, topic: fromIot, bootstrap.servers: kafka.host:9092"
        }
    ]

i use kafka client on centos to check topic, topic is exist, and send messeage succeed. bitnami/kafka:3.5.1

palmshe
asked 8 months ago204 views
1 Answer
0

From the error message, it seems the IoT service can't send a message to Kafka because it believes the topic "fromIot" is not present after waiting for 1000 ms.

Possible Causes:

  1. Connection/Network Issues
  2. Topic Discovery Latency
  3. Kafka Configuration Issues

Diagnostic and Resolution Steps:

1. Test Connectivity:

  • Ensure the IoT service can reach your Kafka cluster:
    telnet kafka.host 9092
  • If you can't connect, there might be network issues or firewall rules blocking the connection.

2. Increase Metadata Timeout:

  • If topic discovery latency is the issue, consider increasing the metadata timeout on the IoT service's Kafka client.

3. Check Kafka Logs:

  • Examine Kafka server logs for any warnings or errors related to the topic "fromIot" or client connections.

4. Kafka Configuration:

  • Verify there are no changes in Kafka configurations that might affect topic creation, metadata, or client connections.

5. IoT Service Configuration:

  • Review the IoT service's Kafka client configuration for any changes, especially concerning bootstrap servers, metadata, and timeouts.

6. Topic Replication:

  • If your topic has multiple replicas, verify all replicas are in sync to avoid metadata discovery issues.

7. Restart Services:

  • Consider restarting the IoT service or Kafka brokers to address potential transient issues.

8. Kafka Client Version:

  • Ensure the Kafka client version used by the IoT service is compatible with your Kafka broker version (bitnami/kafka:3.5.1).

9. Topic Permissions:

  • Ensure the IoT service has the required permissions to produce messages to the "fromIot" topic.

10. Use a Different Client:

  • Test with a different Kafka client to see if the problem persists. This can help determine if the issue is with the IoT service's client or the Kafka setup.
profile picture
EXPERT
AndyAWS
answered 8 months ago
  • thanks! I find EIP Associated to AWS IOT RULE VPC destination is disassociated. Today it is happen again. I feel sad. I don't know Why it happen?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions