Manage and govern Kafka through GitOps - Part 1

Here at Lenses, our vision is to simplify and improve the data-driven journey. To ensure that this road is smooth, repeatable while providing you with confidence we promote GitOps.

What is GitOps?

The core idea is everything is described as configuration. In a set of files and versioned controlled, in git.

Infrastructure as code has become the standard for deploying the base layer of servers and services both on-premise and in the cloud.

At Lenses, we also use this concept. Version control your application and data landscape and have Lenses apply your desired state, using git as the source of truth.

What is Lenses?

Lenses, allows you to build and deploy data flows in a repeatable manner through nothing but declarative configuration over your existing infrastructure, such as Kafka and Kubernetes.

All your data flows come baked-in with enterprise features for monitoring, security and auditing, enabling you to create and troubleshoot your streams. Which means you can get them to production in a fraction of the time, cost and effort.

Why should you care about GitOps?

Git provides an audited and versioned way to manage configuration and make changes to your landscape while adhering to standard development practices.

Declarative configuration is used by Lenses, in the form of YAML or JSON files, to define all aspects of your landscape. For this includes:

• Applications, connectors and processors

• Topics and schemas

• Access controls and quotas

• Alerts and monitoring

• User management and permissions

This means that your entire data landscape: from defining data flows to monitoring, auditing and security can be described and managed in git.

The benefits:

• Deployment of flows in a repeatable fashion with confidence

• Promotion between environments, on-premise to the cloud or from development to production.

• Standardised and familiar workflows for developers, data engineer and release management

• Meet governance - everything audited & version controlled

• Drive automation to accelerate delivery

Check out our hands-on part 2 guide of GitOps with Lenses or download the free Lenses+Kafka all-in-one container to see how managing Kafka doesn’t need to be so difficult:

Manage and govern Kafka through GitOps - Part 2