How To Fix - Partitions Being Revoked and Reassigned issue in Kafka ?
In this Post we will see How To Fix - Partitions Being Revoked and Reassigned issue in Kafka . It appears as a message in the terminal
- "partitions being revoked and reassigned" These normally happen during Rebalancing - Partition or Within Consumer Group. Rebalancing generally means
- Some unstable or unused client
- Unhealthy brokers (due to a variety of reasons)
- Network/OS level issues
When a rebalance happens, there is no consumption which introduces lag. In case , the Error message appears in system with involvement of Consumer Groups , the msg "partitions being revoked and reassigned"- is seen in below 2 scenarios. Assume you have a Consumer group of 3 Consumers and each consumes\processes 1 Kafka broker partitions individually.
Case 1- Suppose consumer 3 goes down for some reason. Subsequently kafka revokes the partition assignment to 3rd consumer temporarily (till it is up and running) and re-assigns the partition to either of consumer 1 or 2. In such kind of scenario , we see such msg "partitions being revoked and reassigned"
Case - 2 Additionally when consumer 3 is up later, it will be re-assigned another partition to consume to process. And you will see the same msg again - "partitions being revoked and reassigned" To Fix it , you need to -
- Check if any consumer within a Consumer group is not working.
- Check all brokers are working
- Any Network/OS level issues
Additional Read -
