How To Fix - KafkaException: Failed to Construct Kafka Consumer

---

This post explains How To Fix - KafkaException: Failed to Construct Kafka Consumer  

Issue:

---

During Execution, sometimes Kafka throws Error Exception message which might look similar . Lets see how can we fix that.

Caused by: org.apache.kafka.common.KafkaException: **Failed to construct kafka consumer**

 

Fix 1:

---

Below are some of the fixes , you should check-back -

• The Kafka Interface ClusterResourceListener ideally should be provided the Latest Kafka Jar (Unless you are using older version of Kafka - Cross check the Compatibility once).
  • Check (pom.xml) if the Kafka jar Dependency used is the RIGHT Version - Upgrade the Jar version if necessary.
  • Also ensure the Kafka Jar is getting added to the Classpath Correctly. Multiple versions of Client jar should not be present.
  • If you are using spring-kafka, cross-check the Version dependency used

 

Fix 2:

---

• Sometimes the issue might also be with Firewall or DNS in BootStrap servers. The Consumer should be able to Reach the Kafka Broker Host.
  • Try pinging the Host to check if any Firewall Blockage.
  • Check if the Cluster Host is accessible from the consumer .

bin/kafka-topics.sh --list --bootstrap-server <HOST\_NAME>:9092

Try the above two fixes. Hopefully you would be able to get rid of the error exception.   Additional Read -

• How To Code a PySpark Cassandra Application ?
• How to Send Large Messages in Kafka ?
• How to Purge a Running Kafka Topic ?

 

Fix - KafkaException: Failed to Construct Kafka Consumer, nifi org.apache.kafka.common.kafkaexception: failed to construct kafka ,consumer, failed to construct kafka consumer spark streaming, failed to construct kafka consumer nifi, failed to construct kafka consumer databricks, exception org apache-kafka common kafkaexception failed to construct kafka producer, failed create new kafkaadminclient, org apache kafka common kafkaexception failed to load ssl keystore, org apache-kafka common kafkaexception java io filenotfoundexception, failed to construct kafka consumer spark streaming, failed to construct kafka consumer nifi, failed to construct kafka consumer databricks, nifi org.apache.kafka.common.kafkaexception: failed to construct kafka consumer, org apache spark sql streaming , streamingqueryexception failed to construct kafka consumer, kafka-clients, failed create new kafkaadminclient, kafka-clients maven,