DevOps | Cloud | Analytics | Open Source | Programming

List of Kafka Commands Cheatsheet

List of Kafka Commands Cheatsheet. In this post we will explore the common kafka commands , kafka consumer group command , kafka command line , kafka consumer command , kafka console consumer command, kafka console producer command .

(We Keep Updating This List - So You can BookMark it)


  Start Zookeeper Services -

bin/ config/

  Start Kafka Server -

bin/ config/

  To Retrieve Replica Assignment Information - This cli script is used to query a list of replicas per log directory on a broker. It provides information for optimizing replica assignment across various brokers in the cluster. --describe --bootstrap-server hostname:port --broker-list broker 1, broker 2 --topic-list topic 1, topic 2


2. Kafka Consumer Commands:

Start Kafka Console Consumer -

bin/ --bootstrap-server localhost:9092 --topic test --from-beginning --max-messages 100

  Write To Topics from Kafka Consumer - Consume from Kafka Broker , process the message and then Write back to Some Other Topic. These need to be part of the Consumer code logic. Below is just a reference using python "kafka" package.

producer.send(Topic1, key = msg.key, value = outmsg1).add\_callback(<SEND\_SUCCESS\_FUNCTION>).add\_errback(<SEND\_ERROR\_FUNCTION>)

  Kafka Consumer Group - --bootstrap-server \[BORKER1\],\[BROKER2\]... --describe --group \[GROUP\_ID\]

  List All the Kafka Consumers - Below command List Consumers that use the Java consumer API i.e. non-Zookeeper-based consumers

bin/ --list --bootstrap-server localhost:9092

  Below command List Consumers that use Zookeeper i.e. those not using the Java consumer API

bin/ --list --zookeeper localhost:2181

  **Get Detail Info about Your Consumer Group - **

bin/ --describe --group <YOUR\_CONSUMER-GROUP-NAME> --bootstrap-server localhost:9092

  View Only 10  Messages on the Terminal -

./ --bootstrap-server localhost:9092 --topic test --from-beginning --max-messages 10


Create Kafka Topic -

bin/ --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic awesome

  Create a Kafka topic if  topic doesn’t exist -

bin/ --zookeeper $ZK\_HOSTS --create --topic test --partitions 3 --replication-factor 3 --**if-not-exists**

  Delete Kafka Topic -

bin/ --zookeeper localhost:2181 --delete --topic test

  List Kafka Topic -

bin/ --list --zookeeper localhost:2181

For Kafka 2.2 and above , you can also use -

bin/ --list --bootstrap-server <BROKER-LIST>

  Topic Properties - This command gives three information -

  • Partition count
  • Replication factor:  '1' for no redundancy and higher for more redundancy.
  • Replicas and in-sync replicas (ISR):  Broker ID’s with partitions and which replicas are current.

bin/ --describe --zookeeper localhost:2181 --topic test

  Increase Topic Partitions - Topic having key, any partition logic or messages order might be impacted . So double check in such scenarios.

bin/ --zookeeper $ZK\_HOSTS --alter --topic test --partitions 5

  Purge a Running Kafka Topic - This is not quite straightforward. So refer our other post for how to do that here - How to Purge a Running Kafka Topic ?    

4. Kafka Spark Application Commands :

Spark-Submit with Kafka Jars - Below command is for submitting a spark python application - which processes Kafka Topic "test" and writes to postgres database. Hence you would need additional jars which needs to be provided through the --jars arguement.

./bin/spark-submit \\

  --packages org.apache.spark:spark-sql-kafka-0-10\_2.11:2.4.1 \\

  --jars spark-sql-kafka-0-10\_2.11-2.4.1.jar,kafka-clients-, postgresql-42.2.10.jar \\ \\   


  subscribe test


5. Kafka Producer Commands :

  Produce a Message to Kafka Topic -

bin/ --broker-list localhost:9092 --topic test


6. Kafka Docker Commands :

  Start the Kafka Docker -

docker-compose up -d

  Attach to the Kafka Broker -

docker exec -it kafka bash


  List all available Kafka brokers in a cluster

./bin/ localhost:2181 ls /brokers/ids

Alternative1 - You can also use -

echo dump | nc -q 2 localhost 2181 | grep brokers

Alternative2 - You can also use - (For Newer versions of Kafka 5.3+)

kafka-broker-api-versions --bootstrap-server BROKER | grep 9092

  Hope you enjoyed this post.   Kafka Command Cheatsheet      

Other Reads -


kafka create topic, kafka commands , kafka consumer group command , kafka command line , kafka consumer command , kafka consumer group, kafka console consumer , kafka console producer , kafka offset , kafka consumer , Kafka producer