Put real-time data to work faster by documenting topic APIs with Kadeck
Data streaming is at the heart of many businesses' real-time infrastructure. Companies use Apache Kafka, Red Panda, and Amazon Kinesis to reliably deliver streaming data from their point of origin to people and systems in real time. The more accessible this data becomes, the more rapidly companies can innovate new user experiences and improve business efficiency via real-time recommendations, vehicle tracking, monitoring & alerting, fraud prevention, and more.
The first step in making real-time streaming data more accessible is to communicate its availability to people, and how to make use of it. This is achieved by documenting message “topics.” Topics and topic message structure compose a business’s real-time API.
Documenting topics and topic message structures is very challenging because there can be very many topics and frequent changes to them. Keeping topic documentation current is time consuming, and unfortunately many organizations fail to update it or even create it in the first place.
In this article we explain who needs to understand the real-time API and how to document it using Kadeck's visual collaboration platform (use it for free) for Apache Kafka, Amazon Kinesis, and Red Panda.
The primary audiences for topic documentation are:
There are multiple reasons why creating API documentation for data streaming topics matters:
Creating documentation for Kafka topics is very straightforward. At a high level, you will want to make it clear:
Below, we present a step-by-step guide based on Apache Kafka Topic API Documentation Template in Kadeck:
Kadeck saves time and assures accuracy by automatically updating many of these template sections, including Schema, Producers and Consumers, and others.
Kadeck is a free, visual collaboration layer for Kafka, Kinesis, and Red Panda, and it makes managing and troubleshooting streaming data easier for teams. Kadeck is comprehensive, and many thousands of people use it to get insights into and manage clusters, topics, consumers, producers, and users.
API documentation is created via the Topic Details page in Kadeck.
A unique advantage to using Kadeck is that a topic’s consumers, producers and schemas are always kept up to date and can be viewed as part of the documentation. This information tends to be outdated quickly in written documents.
If you still want to keep the documentation in tools such as Confluence, it is possible to link to those documents in Kadeck to get the best of both worlds.
One of the standout features of Kadeck is its ability to assign data owners for each Kafka topic. This feature ensures clear lines of responsibility within your team, fostering effective collaboration and communication
As your Kafka topics grow in number, so does the complexity of their management. Kadeck's labeling feature acts as a flexible filing system for your topics, allowing you to categorize, filter, and search them based on functionality, data type, team, or any other relevant metric. This feature refines your workflow, keeping you focused on data management rather than getting lost in the labyrinth of topics.
Kadeck empowers you to craft API documentation for every topic, thereby significantly:
Consider incorporating Kadeck into your streaming data development arsenal today and transform the way you put real-time streaming data to work.