Lenses 5.2: for a healthier Kafka with less manual effort

Kafka adoption is growing fast. Very fast.

At Lenses, we’re pushing out new features to increase developer productivity, reduce manual effort & improve the cost & hygiene of operating your Kafka platform.  Only a few weeks since Lenses 5.1, yet here we are again with more goodies in our release 5.2.

Connector auto restarts

We hear of the same pain pattern: Kafka connect is great for data integration but ensuring all connectors are up and running is manually intensive. 

Most of the time connectors fail because of temporary hiccups that stop tasks. A network interruption or a timeout because the target system is in the middle of a cleanup perhaps? More often than not, nothing that a simple restart cannot fix.

Hopefully you’re already using Lenses to receive alerts when connectors fail. All that’s needed is logging into Lenses and triggering the restart of the Check the task failure and restart.

Now imagine doing this across 10s or even 100s of connectors and tasks, along the day (and night). Not fun.

Enter: Automatic connector restarts. 

You can now effectively handle temporary failures in connectors while also removing the alerting noise. 

Choose how many restarts Lenses will attempt for each failed task before considering it a persistent failure. At that point it will stop restarting that task and send you an alert. 

But Lenses gives you one more point of control: A grace period. That’s because after a restart, connector tasks might take some time to stabilize. Without it, Lenses would be constantly restarting the task reaching the maximum in no time. Give your tasks a few seconds to come to their senses and then Lenses will check if they are well-healthy or need to be restarted again.

Now Lenses quietly restarts failed tasks and only alerts you when it’s a true, persistent failure avoiding alerting false positives

And, if you are worried that the silent restarts are masking a problematic pattern, fear not; Lenses keeps an audit trail of every restart it does. Quickly spot it on Lenses directly, or ship it to Splunk for trend analysis. 

Bulk Topic deletes & UX improvements

Don’t you hate it every time you see a pile of leftover Kafka topics lying around? Wouldn’t it be great if you could just wipe them out in one go? With Lenses 5.2 you can.

Lenses brings bulk deletions. Simply check any topics you want to delete. Mix and match with the Lenses search bar, or quickly select the empty ones with the empty filter. Once you’re happy, hit Actions and Delete. Lenses will let you know which ones you’ve selected and which ones you are allowed to delete so that you get no surprises. Confirm and presto! Clean Kafka in a couple of clicks.

And yes, you can automate it via our API and CLI.

You’ll notice you can now resize columns in our Kafka Topic catalog. Particularly handy for when you have to deal with long topic names. Oh and, go ahead and double click on the column header. Auto-resize. Yes!

Improved Schema Registry Support 

AWS Glue with Protobuf

Rounding off the journey with AWS Glue Schema Registry from 5.1, Lenses now adds support for Protobuf. As always, Glue Protobuf schemas work with both the Lenses SQL Snapshot and SQL Processing engine.

Confluent with Hard deletes

Confluent Schema Registry has 2 ways of deleting schemas: soft and hard.

Lenses 5.2 now supports both. We also switched to hard deletes by default for 2 reasons:

1. It helps keep your platform clean. No leftovers without a reason.

2. Works out-of-the-box with the IBM Event Streams registry

You can always change back to a soft delete by heading to the Schema Registry connection settings in your Lenses.

IBM Event Streams

Lenses now fully supports schemas with IBM Event Streams.

Thanks to our collaboration with IBM, there is now a Confluent-compatible, schema-registry API enabled in the IBM Event Streams.

To learn of all the new features see the readme.

Enjoy the release and remember to connect to our community Slack or our Ask Marios community forum to get the latest info, tips & tricks on Lenses.