DevOps | Cloud | Analytics | Open Source | Programming





How To Fix Kafka Docker Error - "Brokers Does Not Meet The Required Replication Factor"


In this post , we will see How To Fix Kafka Docker Error - "Brokers Does Not Meet The Required Replication Factor" . Replication Factor in Kafka might give error , if it is not properly configured or if there are any discrepancies. This issue especially occurs in Docker implementation of Kafka. See an example of the error below -


Number of alive brokers '1' does not meet the required replication factor '3' for the offsets topic
The issue occurs as the Replication Factor is different from the default value. By default the Kafka Replication Factor is 3. To handle this error, we need to to set the flag - "offsets.topic.replication.factor" to equal 1 in the Kafka configuration. The default value ,as I mentioned, is 3. Assume we have the Kafka configuration as below -


final Properties kafkaConfigs = new Properties();
kafkaConfigs.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, BOOTSTRAP_SERVERS);
kafkaConfigs.put(ConsumerConfig.GROUP_ID_CONFIG, "verifi");
kafkaConfigs.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, Class.forName("org.apache.kafka.common.serialization.StringDeserializer"));
kafkaConfigs.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, Class.forName("org.apache.kafka.common.serialization.StringDeserializer"));
kafkaConfigs.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
To run the Docker version , use the below -  


docker run -d \
--net=host \
--name=kafka \
-e KAFKA_ZOOKEEPER_CONNECT=localhost:32181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:29092 \
-e KAFKA_BROKER_ID=2 \
-e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
confluentinc/cp-kafka:5.5.0
Hope this helps.  

Other Reads  -