Imagine being out on a Saturday morning, drinking coffee and catching some sun when a message on Slack from a colleague tells you about an incident on your Kafka infrastructure. Fortunately, you are able to identify the problem and correct it from where you are but wouldn't it be good to know about that Kafka broker going down at the time that this has happened? Is there a way to prevent that from happening again?
Yes, there is: Lenses alerts.
This tutorial will illustrate how to integrate Lenses and Lenses Alerts with Zapier in order to get email notifications when a Kafka broker is down.
Pre-requisites
In order to be able to follow the steps of this tutorial, you will need the following:
- Lenses up and running. You can use our free Lenses Box instance if necessary.
- An Internet connection, an email address and a Zapier account that allows you to create webhooks and use Zapier email.
The Scenario
We want to get email notifications for certain types of Lenses Alerts, which is this case will be a Kafka broker going offline.
The Implementation
This section will present the steps needed for implementing the described scenario beginning from Lenses.
Note that we are going to connect Lenses and Zapier using Webhooks.
Creating a Zap in Zapier
The first thing that you should do is create a new Zap in Zapier. The Zap will have two components:
- A Zap Webhook that will be used by Lenses.
- A Zap Email that will send an email each time the Webhook is visited.
After logging in to Zapier, select Create Zap
MAKE A ZAP
After that you will have to configure your Zap using existing Zapier components. In this case we will use Webhooks by Zapier
Email by Zapier
Lenses Alert to email - Tsoukalos
After that, select Webhooks by Zapier
Catch Raw Hook
Press on the CONTINUE text in order for Zapier to generate the URL of the webhook, which in this case will be https://hooks.zapier.com/hooks/catch/6433676/ort3dym/
After making sure that you copy and paste the webhook URL for easy access, press on CONTINUE
SKIP TEST
From the next screen, select Email by Zapier
Send Outbound Email
CONTINUE
Although you can set many fields for the email, only the first three are compulsory.
After setting the desired values, click on CONTINUE
SKIP TEST
You are done creating your Zap. The last action you should perform is turning on that Zap by pressing on the relevant switch that appears at the end of your screen and at the upper right corner. After that, press the DONE EDITING
GO TO MY ZAPS
Your newly created Zap should appear on the Zapier list. Press the down sign on the right side of your Zap and select Task History
The Task History
Keep that screen open for easy reference. We are done with Zapier – we will now continue with Lenses.
Creating a Lenses Connection
You will use the Zapier webhook to create a new Lenses Connection from Lenses UI. Go to the Lenses ADMIN
Connections
+ New Connection
Webhook
Based on the address (https://hooks.zapier.com/hooks/catch/6433676/ort3dym/
hooks.zapier.com
Host
Use HTTPS
Please make sure that you do not put HTTP
HTTPS
Host
zapier-broker
Save Connection
Creating a Lenses Channel
Now select Channels
ALERTS
New Alert Channel
Webhook
In the next screen, you will need to fill in the required information, including the Request path
/hooks/catch/6433676/ort3dym/
After that press the Create Alert Channel
zapier-webhook-broker
Using a Channel for getting Alerts
The last step is about connecting that channel with the desired types of alerts – in this case the rule of interest is Kafka Broker is down
Rules
Kafka Broker is down
You are done!
Seeing the results
If a Kafka broker goes down or becomes unreachable, Lenses will use the Zapier webhook as a trigger for sending the mail. The same will happen when a Kafka broker goes up again!
The email message that will be sent, which depends on your configuration, will look as follows:
Additionally, you can visit the Task History
For more technical people
The kind of log messages that you will get for webhooks are as follows:
```
2020-05-01 14:55:41,315 INFO [i.l.c.a.c.w.WebHookAlertPublisher:42] [pool-
12-thread-1] Issuing HTTP request for web hook:
io.lenses.core.alerting.channels.webhook.WebHookChannel@7cc67bd5. Http
Request: HttpRequest(HttpMethod(POST),https://hooks.zapier.com/hooks/catch/6433676/or
t3dym/,List(),HttpEntity.Strict(none/none,ByteString()),HttpProtocol(HTTP/1.
1))
```
Next steps
Now that you know how to generate email alerts from Lenses, you should begin designing your strategy in order to get notified when something happens to your Kafka infrastructure.
