Kafka Topics UI
You are looking at an older version. to view data in Kafka.
Kafka is now the de-facto platform for streaming architectures, and it’s eco-system is maturing, but is not just
Enterprise Ready as many people in Big | Fast Data would like it to be. Lenses.io is a London based start-up
that wants to drive Kafka faster to the future, and thus..
We are announcing the kafka-topics-ui a User Interface that allows browsing data from Kafka Topics and a lot more
The kafka-topics-ui is a user interface that interacts with the Kafka rest-proxy to allow browsing data from Kafka Topics. At the same time it brings visibility by providing a single entry point to explore i) Kafka data, ii) Kafka Schemas, iii) Kafka connectors and a lot more, such as partitions per topic, replication factor per topic and topic configurations.
Kafka REST Proxy
The Kafka REST Proxy - is an excellent
state-less micro-service by Confluent that allows anyone to easily:
- Create a consumer or producer via a POST request
- Consume or Produce Avro | JSON | Binary data on a Kafka Topic
- Delete a consumer
Utilizing it’s capabilities for building a UI was a straight-forward processes after overcoming the initial difficulties. First and foremost we had to tackle the fact you can’t tell a priory what type of data a particular topic contains. Once we figured out that, we can
- hard-code the type of particular config-topics
- automatically detect Avro type by fetching the _schemas topic
- in case type is still unknown iterate through Avro->JSON->Binary consumers till one succeeds
Then things got so much more interesting. Another hidden gem was the flag - that allows us to efficiently stream
just enough data to feed the UI and achieve the above process in just 2-3 seconds.
- Identifies and visualizes Kafka topics and config overrides
- Automatically detects data types (Avro|JSON|Binary)
- Base64 decodes and parses of binary topics
- Provides Table and JSON views
- Allows downloading data from Kafka topics
- Displays number of partitions and replication factor per topic
- Docker container
Check https://github.com/landoop/kafka-topics-ui and follow instructions (Tip: Remember to enable CORS).
The easiest way to get it running is using
docker pull landoop/kafka-topics-ui
And run the container, by specifying the url of the kafka-rest service:
docker run --rm -it -p 8000:8000 \ -e "KAFKA_REST_PROXY_URL=http://localhost:8082" \ landoop/kafka-topics-ui
This is an old version.
to view data in Kafka or the new version for Kafka Topic UI .
Lenses.io - similarly to you continuously brings to light best practices while thriving for operational excellence. Together with the open-source community we are aiming to enhance kafka-topics-ui with new features and capabilities:
- Stream from beginning or real-time capturing
- DC/OS universe components for your Mesos cluster
- Real-time monitoring