DevOps | Cloud | Analytics | Open Source | Programming





How To Fix - "Not all brokers have rack information" Error in Kafka ?


In this post , we will see - How To Fix - "Not all brokers have rack information" Error in Kafka ? Sometimes while trying to create a Kafka topic , we encounter the below error -


Error while executing topic command : Not all brokers have rack information. Add --disable-rack-aware in command line to make replica assignment without rack information.
ERROR kafka.admin.AdminOperationException: Not all brokers have rack information.
This error pertains to configuring broker.rack setting in some of the brokers and missing in some . Manuel topic creation uses this info , if set-up, whereas Automatic process doesn't. Please note , the Rack concept in Kafka is very similar in HDFS rack concept. Which means this feature could be useful ONLY if you distribute , tag and map the Kafka brokers in your cluster across various racks. If you define rack for each of your Kafka Broker nodes , then Kafka uses those information to create replicas on nodes that do not share the same rack.This provides the same advantage as that of HDFS file system - fault-tolerance. More details on this can be found in the Kafka Official documentation - http://kafka.apache.org/documentation.html#basic_ops_racks To fix the issue , use any of the below options -

  • Ignore the Rack awareness info while creating the topic manually - use the flag as shown below
 


/bin/kafka-topics.sh --create --zookeeper localhost:2181 **\--disable-rack-aware** --replication-factor 1 --partitions 1 --topic TEST\_TOPIC
 

  • Go to Kafka server.properties file and find the broker.rack flag. Either set it to null (If you want to disable rack info) or use some sort of id (if you want to use the rack info).
  Hopefully this will solve the problem.   Other Reads -