Antonios Chalkiopoulos
For too long engineering teams have been fighting technical and operational problems as well as suffering from a knowledge shortage whilst building data products. That’s time away from addressing business requirements.
It always shocks us to hear companies trying to deliver real-time applications, spending on average 9 months before they reach production. So our mission at Lenses has been clear, to promote DataOps best-practices and remove all the friction so that projects can be delivered in a fraction of that time with confidence and security.
Never worry about data infrastructure
Data privacy and data ethics out of the box
A secure access layer for data and applications
Data landscape management across all data technologies via GitOps
Zero-setup SQL and Streaming SQL for low-code real-time applications
What drives us forward is the adoption and enthusiasm we receive: “Lenses turns the lights on our real-time data and apps” data engineers reporting.
This is how we enable some of the largest brands and innovating startups worldwide to accelerate the adoption of real-time practices and deliver new digital experiences, including Daimler, Zopa and Babylon Health and many others.
As the real-time streaming data platform expands beyond departments and adoption is growing across every organisation, two major challenges appear:
How to control and govern data privacy while empowering many users and applications?
How to rapidly enable even non-technical people to access insights and collaborate more with the engineers?
This new release addresses the above challenges by providing a secure gateway that promotes collaboration.
Namespace and fine role based access introduce the concept of data ownership and enable seamless multi-tenant capabilities on Apache Kafka. A data namespace identifies one or more topics (or wildcards) and allows assigning permissions, such as create, query, update topics or schemas.
Beyond the data layer, namespaces expand to the application layer. Who can control Kafka connectors, pipelines, topologies, manage consumers and use Streaming SQL. And it’s not just about people, but also service accounts for GitOps and CI/CD purposes.
Auditing has always been a by-design capability, and at this new version, it expands to all data access (SQL queries) as well as user logins. This is particularly useful if you have security and compliance controls to meet.
And finally, beyond a pluggable authentication (so one can plug-in any authentication provider) Lenses now supports cascading & multiple authentication strategies. Imagine for example authenticating internal users via Kerberos and external partners via LDAP setup at the same time!
When having companies with hundreds or thousands of users on Lenses, it’s normal to collect a lot of feedback, and every change improving a large audience. In this new major release we have revamped our UI and experience to separate clearly users functions and admin functions.
This further simplifies the experience of users and increases the productivity.
Lenses provides data and application performance monitoring and alerting for real-time data. With this new release, the alerting mechanism is now pluggable. What if I want an infrastructure alert to automatically generate a ticket or notify my in-house incident management solution?
With such a large ecosystem to cover, we decided to deliver a pluggable framework, and open-source plugin integrations, so that the ecosystem can easily grow. Of course, we are targeting of providing such alert plugins for most popular systems ie. PagerDuty with Slack and AlertManager being already available. With this new feature, one can easily create and use his own custom plugin via a simple process.
Connecting Lenses to a managed Kafka service such as AWS MSK, Azure HDInsight, Confluent Cloud and others is now even easier. Auto-configuration and auto-detection now makes it a process of a few clicks to install Lenses on top of MSK or Azure and better integrates by supporting TLS setups out of the box.
You’ll notice many SQL and Streaming SQL improvements including new features, a help command to assist users and better error-handling across any data format.
Extending and integrating Lenses is achievable via Lenses API which now has a new public doc or the open-source Lenses CLI. Improvements are also available in the Python library as well as the SQL JDBC driver which enables seamless data access to real-time data ie. for 3rd party BI tools.
We launched the DataOps learning center with additional information that can get anyone started with a much easier experience. We are happy for our team, customers and the wider community to share tips, best practices, guides and mind share
Full release notes can be found here
Enjoy the release!
Want to try 3.0 for yourself: Get Started here
For existing customers: For 3.0 updates please use the welcome area
Subscribe to our Webcast of Lenses on 17th October showcasing Lenses 3.0!
Meet the SRE AI Agent for Apache Kafka....
Andrew Stevenson
