DevOps | Cloud | Analytics | Open Source | Programming





How to Get Count of Messages in a Kafka Topic ?


In this post , we will see How to Get Count of Messages in a Kafka Topic. There are different ways and workarounds to get this done. However do note that these options might not be an Exact count of messages and could turn out to be a ballpark estimate.  

Option 1: Using ConsumerOffsetChecker

If you are using an older version of Kafka , then you could use ConsumerOffsetChecker concept for this.


kafka-run-class.sh kafka.tools.ConsumerOffsetChecker \\
--topic <TOPIC\_NAME> \\
--zookeeper localhost:2181 \\
--group <GROUP\_NAME>
 

Option 2: Using ConsumerGroupCommand

However if you are using a newer version of Kafka which has no support for ConsumerOffsetChecker , you could try using ConsumerGroupCommand as shown below -


kafka-run-class.sh kafka.admin.ConsumerGroupCommand \\
--group <GROUP\_NAME> \\
--bootstrap-server localhost:9092 \\
--describe
The output will look somewhat like below . The "LAG" will indicate the COUNT of messages in the specific partition within a Topic.


TOPIC     PARTITION  CURRENT-OFFSET  LOG-END-OFFSET   LAG
TEST      0          XXxxx             YYyyyy         ZZ1 
TEST      1          XXxxx           YYyyyy           ZZ2
TEST      2          XXxxxx           YYyyyy          ZZ3
 

Option 3: Using GetOffsetShell

If you are using newer version of Kafka , you can try the below option of kafka.tools.GetOffsetShell as well.


$ kafka-run-class kafka.tools.GetOffsetShell \\
--broker-list <HOST1:PORT,HOST2:PORT> \\
--topic <TOPIC\_NAME>
This command will display the number of messages in each Topic Partitions.  

Option 4: Using Kafkacat

You can also use Kafkacat to count the no. of messages or events.


$ kafkacat -b broker:<PORT> -t <TOPIC\_NAME> -C -e -q| wc -l
 

Option 5 : Using KSqlDB

Let's say you have created a schema of Kafka Topic "Test1" in the KsqlDB.


CREATE STREAM ST1 (Date VARCHAR ,
                Time VARCHAR ,
                Field1 VARCHAR )
WITH (KAFKA\_TOPIC='TEST1',
VALUE\_FORMAT='DELIMITED');
Next based on whatever value you store in the flag 'auto.offset.reset' , accordingly it will consider the messages to be considered. "earliest" will indicate to read all the messages. When you execute the below sql, it will display the total count of messages based on the offset preference set with 'auto.offset.reset'.


SELECT NAME, COUNT(\*) AS Count\_Of\_Msg
FROM ST1
GROUP BY 1
EMIT CHANGES;
 

Option 6 : Using kafka-console-consumer.sh

You can also use kafka-console-consumer.sh script to count the messages.


kafka-console-consumer.sh \\
--from-beginning \\
--bootstrap-server <BROKER:PORT> \\
--property print.key=true \\
--property print.value=false \\
--property print.partition \\
--topic <TOPIC\_NAME> | tail -n 10|grep "Msg Count="
  Hope this helps.  

**Other Interesting Reads - **

   


kafka count number of messages in topic ,kafka count messages in topic ,kafka count messages in topic cli ,kafka count messages in topic python ,kafka count events in topic ,kafka count records in topic ,kafka count messages ,kafka count messages in partition ,kafka count number of messages ,kafka total message ,kafka total messages in topic ,kafka total messages behind ,kafka total message count ,kafka messages in total ,how to see the messages in kafka topic ,how to count messages in kafka topic ,how to check total messages in kafka topic ,how to get all messages from kafka topic ,kafka topic total message count ,kafka get total message count ,kafka total number of messages ,kafka count messages ,kafka list message count ,kafka get message count ,how to check kafka topic message count ,how to count number of messages in kafka topic ,count total messages in kafka topic ,count total messages in kafka topic java ,kafka message count in topic ,get total messages in kafka topic ,how to check the messages in kafka topic ,how to see messages in kafka topic ,total number of messages in kafka topic ,how to get the number of messages in a kafka topic ,kafka consumer get total message count ,kafka server brokertopicmetrics messages in total ,kafka count message ,kafka count messages in topic ,kafka count messages in topic cli ,kafka count messages in topic python ,kafka message count limit ,kafka messagesinpersec count ,kafka streams count messages ,java kafka count messages ,kafka count number of messages in topic ,kafka count number of messages ,kafka count messages ,kafka topic message count command ,kafka check message count in topic ,kafka get message count ,how to count kafka messages ,how to check the messages in kafka topic ,how to see the messages in kafka topic ,how to count number of messages in kafka topic ,java kafka message count ,kafka list messages count in topic ,kafka list message count ,kafka message length limit ,kafka max message count ,kafka manager messages count ,kafka get count of messages in topic ,kafka message count per topic ,prometheus kafka message count ,kafka producer message count ,how to check kafka topic message count ,kafka queue message count ,kafka message retry count ,kafka count message topic ,kafka topic message count limit ,kafka total message count ,kafka view message count ,count total messages in kafka topic java ,kafka count messages in a topic ,kafka count number of messages in a topic ,kafka message count in topic ,kafka console count messages in topic ,



kafka command line count messages in topic ,kafka get message count in topic ,how to count messages in kafka topic ,kafka total messages in topic ,kafka count messages on topic ,kafka count messages in topics ,count total messages in kafka topic ,kafka queue size limit ,maximum size of message in kafka ,kafka maximum topic count ,kafka max topic count ,kafka topic number limit ,kafka.server.brokertopicmetrics.messagesinpersec.count ,how to check total messages in kafka topic ,how to count messages in kafka topic ,how to get all messages from kafka topic ,how to see the messages in kafka topic ,kafka how to total message action ,kafka how to total message box ,kafka how to total message error ,kafka how to total message event ,kafka how to total message example ,kafka how to total message exception ,kafka how to total message group ,kafka how to total message handler ,kafka how to total message header ,kafka how to total message hooks ,kafka how to total message java ,kafka how to total message job ,kafka how to total message json ,kafka how to total message key ,kafka how to total message length ,kafka how to total message load ,kafka how to total message logs ,kafka how to total message long ,kafka how to total message messages ,kafka how to total message methods ,kafka how to total message mode ,kafka how to total message name ,kafka how to total message node ,kafka how to total message null ,kafka how to total message object ,kafka how to total message output ,kafka how to total message parameter ,kafka how to total message parameters ,kafka how to total message path ,kafka how to total message query ,kafka how to total message query params ,kafka how to total message questions ,kafka how to total message queue ,kafka how to total message request ,kafka how to total message response ,kafka how to total message solution ,kafka how to total message type ,kafka how to total message update ,kafka how to total message url ,kafka how to total message video ,kafka how to total message xampp ,kafka how to total message xcode ,kafka how to total message xml ,kafka how to total message xpath ,kafka how to total message yaml ,kafka how to total message you ,kafka how to total message zero ,kafka how to total message zip ,kafka how to total message zone ,kafka messages in total ,kafka topic total message count ,kafka total message count ,kafka total messages behind ,kafka total messages in topic ,how to check kafka topic message count ,how to check the messages in kafka topic ,



how to check total messages in kafka topic ,how to count messages in kafka topic ,how to count number of messages in kafka topic ,how to get all messages from kafka topic ,how to see messages in kafka topic ,how to see the messages in kafka topic ,kafka consumer processed a total of 0 messages ,kafka count messages ,kafka get message count ,kafka get total message count ,kafka list message count ,kafka message reader tool ,kafka message viewer tool ,kafka messages in total ,kafka tool delete messages ,kafka tool message ,kafka tool message format ,kafka tool message headers ,kafka tool not showing messages ,kafka tool to view messages ,kafka topic total message count ,kafka total message authentication ,kafka total message bar ,kafka total message box ,kafka total message count ,kafka total message docker ,kafka total message error ,kafka total message event ,kafka total message example ,kafka total message handler ,kafka total message header ,kafka total message java ,kafka total message json ,kafka total message key ,kafka total message kubernetes ,kafka total message linux ,kafka total message list ,kafka total message manager ,kafka total message meaning ,kafka total message method ,kafka total message node ,kafka total message not found ,kafka total message not working ,kafka total message pack ,kafka total message plugin ,kafka total message query ,kafka total message questions ,kafka total message request ,kafka total message required ,kafka total message response ,kafka total message update ,kafka total message upload ,kafka total message validation ,kafka total message value ,kafka total message variable ,kafka total message warning ,kafka total message writing ,kafka total message xcode ,



kafka total message xml ,kafka total message xpath ,kafka total message yaml ,kafka total message you ,kafka total message youtube ,kafka total message zip ,kafka total message zone ,kafka total message zoom ,kafka total messages behind ,kafka total messages in topic ,kafka total number of messages ,how to check kafka messages ,how to check kafka topic message count ,how to check message count in kafka topic ,how to check message on kafka topic ,how to check the messages in kafka topic ,how to count messages in kafka topic ,how to count number of messages in a kafka topic ,how to count number of messages in kafka topic ,how to get kafka message count ,how to get kafka message size ,how to get the number of messages in a kafka topic ,how to see kafka messages ,how to see the messages in kafka topic ,kafka count message in topic ,kafka count message topic ,kafka count messages in topic ,kafka count messages in topic cli ,kafka count messages in topic python ,kafka count messages on topic ,kafka how to count message box ,kafka how to count message by class ,kafka how to count message error ,kafka how to count message event ,kafka how to count message exception ,kafka how to count message facebook ,kafka how to count message given ,kafka how to count message greater than ,kafka how to count message groups ,kafka how to count message header ,kafka how to count message history ,kafka how to count message hooks ,kafka how to count message messages ,kafka how to count message messenger ,kafka how to count message mode ,kafka how to count message name ,kafka how to count message node ,kafka how to count message number ,kafka how to count message query ,kafka how to count message questions ,kafka how to count message queue ,kafka how to count message update ,kafka how to count message url ,kafka how to count message users ,kafka how to count message video ,kafka how to count message yaml ,kafka how to count message year ,kafka how to count message yesterday ,kafka how to count message you ,kafka how to count message youtube ,kafka how to count message zero ,kafka how to count message zeros ,kafka how to count message zone ,kafka message count limit ,kafka message count per topic ,kafka message length limit ,kafka messagesinpersec count ,kafka streams count messages ,kafka topic message count command ,best kafka count messages ,can kafka count messages ,confluent kafka topic message count ,count the number of messages in a kafka topic ,did kafka count messages ,do kafka count messages ,e kafka count messages ,how to check kafka topic message count ,how to check message count in kafka topic ,how to check message on kafka topic ,how to check the messages in kafka topic ,how to count kafka messages ,how to count messages in kafka topic ,



how to count number of messages in kafka topic ,how to get the number of messages in a kafka topic ,how to see the messages in kafka topic ,java kafka count messages ,java kafka message count ,kafka check message count in topic ,kafka count message by date ,kafka count message deprecated ,kafka count message docker ,kafka count message error ,kafka count message example ,kafka count message topic ,kafka count message url ,kafka count message whatsapp ,kafka count message yaml ,kafka count message year ,kafka count message yearly ,kafka count message youtube ,kafka count messages in a topic ,kafka count messages in topic ,kafka count messages in topic cli ,kafka count messages in topic python ,kafka count number of messages ,kafka count number of messages in topic ,kafka get count of messages in topic ,kafka get message count ,kafka list message count ,kafka list messages count in topic ,kafka manager messages count ,kafka max message count ,kafka message count limit ,kafka message count per topic ,kafka message length limit ,kafka message retry count ,kafka messagesinpersec count ,kafka producer message count ,kafka queue message count ,kafka show message count in topic ,kafka streams count messages ,kafka topic check message count ,kafka topic message count command ,kafka topic message count limit ,kafka total message count ,kafka view message count ,kafka-topics message count ,prometheus kafka message count ,python kafka consumer message count ,should kafka count messages ,top kafka count messages ,u kafka count messages ,w kafka count messages ,was kafka count messages ,what kafka count messages ,when kafka count messages ,where kafka count messages ,will kafka count messages ,worst kafka count messages ,would kafka count messages ,x kafka count messages ,y kafka count messages ,z kafka count messages