Learn DataOps in the Lenses Kafka Docker Box
Lenses Box is a complete container solution for you to build applications on a localhost Apache Kafka docker.
Enjoy a 3-min tour!
01.
What is included
Apache Kafka v. 2.5.1, Elasticsearch v. 6.8.7 and ecosystem of open source kafka tools and real-time example data.
Here are some of the things you’ll be able to do:
Develop real-time applications
Monitor Kafka and app flows
Run admin Kafka operations
Explore streaming data with SQL
Protect HIPAA,GDPR,PCI data
Use streaming SQL applications
Manage schemas, consumers and more
02.
Start
First get your key heredocker run -e ADV_HOST=127.0.0.1 \
-e EULA="https://licenses.lenses.io/d/?id=REGISTER_FOR_KEY" \
--rm -p 3030:3030 -p 9092:9092 lensesio/box:latest
Note: Make sure you receive in your email your free access key
Note: Kafka Docker will require 4GB RAM. See how to increase docker RAM
TIP
If you run on a VM and want Kafka to be accessible over the Internet, set as ADV_HOST your public IP address.
04.
Explore real-time data
2. Run the following query:
SELECT Speed, Latitude, Longitude
FROM sea_vessel_position_reports
WHERE Speed > 10
05.
Create a new Kafka topic
2. Use the New Topic button
TIP
You can also automate via GitOps
06.
Create a data pipeline
1. Navigate to Connectors
2. Click New Connector and select the file source connector
3. Use the following configuration
name=my-real-time-file-connector
connector.class=org.apache.kafka.connect.file.FileStreamSourceConnector
topic=broker_logs
file=/var/log/broker.log
tasks.max=1
This will continuously move log events from /var/log/broker.log into the kafka topic.
KAFKA PRODUCER
To produce messages with a Kafka producer use as bootstrap broker your ADV_HOST and port 9092. eg. PLAINTEXT://127.0.0.1:9092
07.
Stream processing
Use streaming SQL to continuously process real-time data.
2. Set a name and use streaming SQL
SET defaults.topic.autocreate=true;
INSERT INTO speed_boats
SELECT STREAM MMSI, Speed, Longitude AS Long, Latitude AS Lat, `Timestamp`
FROM sea_vessel_position_reports
WHERE Speed > 10;
TIP
08.
Consumer Monitor
TIP
When a Consumer is not running you can also change the offsets!
09.
Monitor real-time flows
realtime mapof how applications ,connectors and streaming SQL interacts with Kafka topics.Click on a node to see the details, metrics and health statuses.
TIP
You can add your applications, data pipelines and also microservices in the topology map.
10.
Setup alerting
Infrastructure health
Producer SLAs
Consumer SLAs
Explore advanced options
Data centric security
Single Sign On authentication
HIPAA,GDPR,PCI data policies
Automate Flows with GitOps
Manage Kafka ACLs & Quotas
AWS MSK, Azure and other Cloud integrations
Frequently Asked Questions
Yes.
Lenses Box is optimized for development and includes a single broker Kafka environment. However, you can evaluate Lenses against your existing setup. Lenses works with all popular Kafka distributions and clouds. Contact our team to help youLenses Box is Free for ALL and forever.
In order to use the Docker of Lenses Box you require a free access key. You may need to refresh your key from time to time for security reasons.
You will need a paid subscription to connect Lenses to your own cluster or production.
See pricingIt's not recommended.
Lenses Box is a single-node setup. For production environments it is recommended to have a multi-node setup for scalability and fail-over.
For production or multi-node setups of your Kafka cluster contact our team
Yes. Lenses works with AWS MSK, Azure HDInside, Aiven, Confluent Cloud and other managed cloud services.
Lenses also provides deployment templates tailored for each Cloud provider.
You can use Lenses to Monitor, Explore, Build and Secure your data platform.
Check how our customers use Lenses →Let us help