DevOps | Cloud | Analytics | Open Source | Programming

How To Fix - Partitions Being Revoked and Reassigned issue in Kafka ?

In this Post we will see How To Fix - Partitions Being Revoked and Reassigned issue in Kafka . It appears as a message in the terminal

  • "partitions being revoked and reassigned" These normally happen during Rebalancing - Partition or Within Consumer Group. Rebalancing generally means
  • Some unstable or unused client
  • Unhealthy brokers (due to a variety of reasons)
  • Network/OS level issues
When a rebalance happens, there is no consumption which introduces lag. In case , the Error message appears in system with involvement of Consumer Groups , the msg "partitions being revoked and reassigned"- is seen in below 2 scenarios. Assume you have a Consumer group of 3 Consumers and each consumes\processes 1 Kafka broker partitions individually. Case 1- Suppose consumer 3 goes down for some reason. Subsequently kafka revokes the partition assignment to 3rd consumer temporarily (till it is up and running) and re-assigns the partition to either of consumer 1 or 2. In such kind of scenario , we see such msg "partitions being revoked and reassigned" Case - 2 Additionally when consumer 3 is up later, it will be re-assigned another partition to consume to process. And you will see the same msg again - "partitions being revoked and reassigned"   To Fix it , you need to -

  • Check if any consumer within a Consumer group is not working.
  • Check all brokers are working
  • Any Network/OS level issues
    Additional Read -