Kafka Topics UI

Hey there, You are looking at an older version. Go to Lenses to view data in Kafka.

Kafka is now the de-facto platform for streaming architectures, and it’s eco-system is maturing, but is not just yet as Enterprise Ready as many people in Big | Fast Data would like it to be. Lenses.io is a London based start-up that wants to drive Kafka faster to the future, and thus…

We are announcing the kafka-topics-ui a User Interface that allows browsing data from Kafka Topics and a lot more

About Kafka-Topics-UI

The kafka-topics-ui is a user interface that interacts with the Kafka rest-proxy to allow browsing data from Kafka Topics. At the same time it brings visibility by providing a single entry point to explore i) Kafka data, ii) Kafka Schemas, iii) Kafka connectors and a lot more, such as partitions per topic, replication factor per topic and topic configurations.

Kafka REST Proxy

The Kafka REST Proxy - is an excellent state-less micro-service by Confluent that allows anyone to easily:

• Create a consumer or producer via a POST request

• Consume or Produce Avro | JSON | Binary data on a Kafka Topic

• Delete a consumer

Utilizing it’s capabilities for building a UI was a straight-forward processes after overcoming the initial difficulties. First and foremost we had to tackle the fact you can’t tell a priory what type of data a particular topic contains. Once we figured out that, we can

1. hard-code the type of particular config-topics

2. automatically detect Avro type by fetching the _schemas topic

3. in case type is still unknown iterate through Avro->JSON->Binary consumers till one succeeds

Then things got so much more interesting. Another hidden gem was the flag - that allows us to efficiently stream just enough data to feed the UI and achieve the above process in just 2-3 seconds.

?max_bytes=50000

Kafka-Topics-UI features

Currently

• Identifies and visualizes Kafka topics and config overrides

• Automatically detects data types (Avro|JSON|Binary)

• Base64 decodes and parses of binary topics

• Provides Table and JSON views

• Allows downloading data from Kafka topics

• Displays number of partitions and replication factor per topic

• Docker container

Using it

Check https://github.com/landoop/kafka-topics-ui and follow instructions (Tip: Remember to enable CORS). The easiest way to get it running is using docker:

docker pull landoop/kafka-topics-ui

And run the container, by specifying the url of the kafka-rest service:

Quick preview

Conclusions

Hey there, This is an old version. Check Lenses to view data in Kafka or the new version for Kafka Topic UI here.

Lenses.io - similarly to you continuously brings to light best practices while thriving for operational excellence. Together with the open-source community we are aiming to enhance kafka-topics-ui with new features and capabilities:

• Stream from beginning or real-time capturing

• DC/OS universe components for your Mesos cluster

• Real-time monitoring

Find out more Tools

Lenses for Kafka

Lenses Box, Kafka development environment (topics, schemas, connectors & Kafka all-in-one!)

Tutorials

• Kafka Connect Pipelines, sink to Elasticsearch

• Time-Series with Kafka, Kafka Connect & InfluxDB

• From MQTT to Kafka with Connect and Stream Reactor