How To Fix - Multiple Broker Start Error in Kafka ?

---

In this post , we will see - How To Fix - Multiple Broker Start Error in Kafka. You might face issue while trying to start multiple Broker in a Kafka cluster.

ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentBrokerIdException: Configured broker.id 2 doesn't match stored broker.id 1 in meta.properties. If you moved your data, make sure your configured broker.id matches. If you intend to create a new broker, you should remove all data in your data directories (log.dirs).

INFO shutting down (kafka.server.KafkaServer)

  These types of issues are more often than not related to the configuration settings. However we will check all the relevant pointers one by one. When you face the same  error , try the below fixes and see if that helps.  

Starting Kafka:

• When you start the Kafka server , use respective config files for each brokers.

kafka/bin/kafka-server-start.sh \\
kafka/config/server.properties

 

Zookeeper side:

• Flush out the data in the data directories (log.dirs).
• Subsequently check if zookeeper is running fine

 

Kafka Config File:

• Ensure you are not using any Duplicates while you configure each Broker settings.
• Cross check the Port and log.dir used in the config file against each of the kafka brokers in the cluster.
• Cross check the config.listeners and advertised.listeners values for each broker if these are correctly put.

Below is a sample config file with parms

config.listeners=PLAINTEXT://<Broker\_ip:<Port>
advertised.listeners=PLAINTEXT://<Broker\_ip:<Port>

 

• Use different log directories for each of the brokers to keep the logging clean and segregated.

log.dirs=/tmp/log-dir-1

Also read our other post on similar line for additional info  - here   Try out these and see if that helps.  

Other Interesting Reads -

• How to Enable UTF-8 in Python ?

• How to log an error in Python ?

• Sample Python Code To Read & Write Various File Formats (JSON, XML, CSV, Text)

• How to Handle Errors and Exceptions in Python ?

• How to Handle Bad or Corrupt records in Apache Spark ?

• How To Fix – Partitions Being Revoked and Reassigned issue in Kafka ?

• What Are The Most Important Metrics to Monitor in Kafka ?

• How To Connect Local Python to Kafka on AWS EC2 ?

   

kafka multi broker cluster setup ,kafka multi node cluster setup ,kafka multiple brokers single node ,kafka meta properties ,multi node multi broker kafka ,kafka zookeeper ,apache kafka ,kafka broker ,Multiple Broker Kafka ,multi broker kafka ,multi broker kafka docker ,multi broker kafka setup ,multiple brokers kafka ,multiple brokers in kafka ,multi broker kafka cluster ,kafka single node multiple broker configuration ,kafka broker multiple disks ,multiple broker in kafka ,kafka broker multiple listeners ,multi node multi broker kafka ,multiple kafka broker on single node ,kafka-node multiple broker ,running multiple kafka brokers on same server ,kafka one broker multiple partition ,kafka broker multiple ports ,kafka single broker multiple partitions ,kafka multiple broker setup ,kafka multi broker docker compose ,kafka multi broker configuration ,wurstmeister kafka multi broker ,kafka multi broker cluster setup ,multiple kafka brokers on single node ,running multiple kafka brokers ,start multiple kafka brokers ,kafka multiple brokers docker ,kafkacat multiple brokers ,kafka multiple brokers example ,kafka multiple brokers kubernetes ,kafka multiple broker configuration ,apache kafka multiple brokers ,spring boot multiple kafka brokers ,kafka consumer multiple brokers ,create multiple brokers in kafka ,confluent kafka multiple brokers ,kafka connect multiple brokers ,camel kafka multiple brokers ,kafka cluster multiple brokers ,embedded kafka multiple brokers ,how to start multiple brokers in kafka ,why we need multiple brokers in kafka ,how to use multiple brokers in kafka ,kafka listeners multiple brokers ,kafka node multiple brokers ,kafka multiple brokers partitions ,kafka producer multiple brokers ,python kafka multiple brokers ,spring kafka multiple brokers ,kafka streams multiple brokers ,kafka with multiple brokers ,how many brokers in a kafka cluster ,kafka how many brokers ,how to create multiple brokers in kafka ,how to setup multiple brokers in kafka ,how many kafka brokers do i need ,how many kafka brokers should i have ,how to run multiple brokers in kafka ,setting up a multi-broker cluster kafka ,create a kafka multi-broker cluster ,apache kafka multi broker cluster with multi node zookeeper ,kafka multiple brokers single node ,