How to reset consumer offsets in Apache Kafka
Change consumer offsets with the command line or Kadeck
If the pipeline stops because of a failed test or faulty records, it is necessary to reset the offsets of a consumer or consumer group. In this article, you will learn how to do this either with the command line or via the KaDeck interface.
This is a blog post from our Community Stream: by developers, for developers. Don’t miss to stop by our community to find similar articles or join the conversation.
Be it a failed test case, faulty data records, or an error in the consuming application itself: if the consumer does not continue, the data pipeline stops, and all downstream processes no longer receive any data. In such a case, there is really only one option: the data records that lead to the problem must be skipped.
For this purpose, you can use the console application supplied with Apache Kafka, provided that you have direct access to the cluster and the appropriate rights. However, if you want to conveniently monitor the consumer and change offsets via an interface, the use of KaDeck is a good choice.
Whichever way you choose, in this article we discuss both approaches.
If you are already using KaDeck or are interested in how intuitively consumer offsets can be managed in KaDeck, we have created a short video about this.
How do I find out which consumer is hanging?
If you have already located your problem child, you can safely skip this section.
Since version 2.4.0 of Apache Kafka, there is the possibility to check a list of all consumer groups and their lag via the console application “kafka-consumer-groups“. The command for this is:
If you are not yet running version 2.4.0 or later or do not have access via the console application, the -all-groups addition is not available, and you should rely on third-party applications to solve this use case efficiently.
If you already know the consumer group, you can slightly modify the above command to retrieve the lag of a known consumer group (in the example: “myConsumerGroup“):
Through KaDeck, there are several ways to see faulty or stalled consumers. If you are using the Enterprise version of KaDeck Web, you can also create alerts to notify you of slow or stalled consumers.
Select the desired topic in the Data Browser and click Flow. If your consumer is already inactive, you have to activate the option “show inactive” (eye icon on the right). Slow consumer groups are displayed in yellow, stopped consumer groups in red. If you move the mouse pointer over a consumer, the offset lag is also displayed.
Reset consumer offsets
Once you have found the consumer or consumer group that you want to reset, you must make sure that the consumer group is not active: all applications in the consumer group must therefore be shut down. Otherwise, the offsets of a consumer or consumer group cannot be changed.
Using the console application “kafka-consumer-groups“, reset the offsets as follows:
Instead of –to-earliest, which causes the consumer to be set back to the beginning, –to-latest can be used to set the consumer to the end of the stream. Thus, all messages not consumed so far will be skipped. Alternatively, there is still the possibility to specify a time:
In KaDeck, the offsets of a single consumer or the complete consumer group can be managed via the “Stream Details” view.
How to do it with KaDeck
In this short video, you’ll learn how to reset the offsets of a stuck consumer with just a few clicks in KaDeck.
Conclusion
In this short blog article, we have shown how to manage the offsets of a consumer using the console application provided with Apache Kafka. On the other hand, we briefly touched on the possibilities in KaDeck. Be sure to watch our video, which shows a typical scenario.
Did you like this article? Feel free to let me know (Twitter: @benjaminbuick or the Xeotek team via @xeotekgmbh)!
Ben
Software architect and engineer with a passion for data streaming and distributed systems.
