By Mihalis Tsoukalos | Jun 01, 2020


Kafka SMS health alerts via Zapier integration

Kafka SMS health alerts via Zapier integration

Systems fail from time to time. And there is nothing worse than being unaware a system is down at all. Which is why we wrote the email alerts from Lenses.io tutorial.

But what about those evenings where you are enjoying a documentary or just having an email-free evening? After all, system failures rarely happen at a magically convenient time.

This is why we have created this tutorial, which will illustrate how to integrate Lenses Alerts with Zapier to get SMS messages directly to your phone when one of your Kafka brokers is down.

Pre-requisites

In order to be able to follow the steps of this tutorial, you will need the following:

  • Lenses up and running. You can use our free Lenses Box instance if necessary.

  • An Internet connection and a Zapier account that will allow you to create a webhook and use Zapier SMS. As Zapier currently supports USA and UK mobile numbers only, you will also need to have a USA or UK mobile number. In our case, we will use a UK mobile number.

The Scenario

We want to get SMS notifications for certain types of Lenses Alerts, which is this case will be a Kafka broker going offline.

Note that the same process will allow you to get SMS alerts for all the available Lenses Infrastructure alert rules.

The Implementation

This section will present the steps needed for implementing the described scenario beginning from Lenses.

Creating a Zap in Zapier

The first thing that you should do is create a new Zap in Zapier. The Zap will have two components:

  • Zap Webhook that will be used by Lenses.

  • Zap SMS that will send an SMS each time the Webhook is visited. You will associate that Zap SMS app with the desired mobile number in a while.

After logging in to Zapier, select Create Zap or MAKE A ZAP from your Zapier screen.

Create Zap

After that you will have to configure your Zap using existing Zapier components.

At this point it would also be a good idea to name you Zap by putting some text in the upper left corner. We will name our Zap Lenses Alert to SMS - (Tsoukalos). Although you can use any description you want, it is good to give a descriptive name followed by your name or surname in order to be able to find your Zap easily.

After that, select Webhooks by Zapier, which will take you to the next screen. The name of the App will automatically appear on the screen – for that app you will have to select the Trigger Event, which in this case will be Catch Raw Hook.

Create Zapier WebHook (SMS)

Press on the CONTINUE text in order for Zapier to generate the URL of the webhook, which in this case will be https://hooks.zapier.com/hooks/catch/6433676/oiooeim/.

A Zapier Web

After making sure that you copy and paste the webhook URL for easy access, press on CONTINUE. As it would be difficult to test the webhook at this phase, press Skip Test on the screen in order to continue, select and configure the next component of your pipeline.

From the next screen, select SMS by Zapier from the Built-In Apps or Your Apps group. This will take you to the next screen.

SMS by Zapier

If this is the first time that you are using SMS by Zapier, you will have to connect a mobile number to that app for reasons of security. Press Sign in to SMS by Zapier to continue to the next screen:

Access SMS number

For now only US and UK numbers are supported by SMS by Zapier app – we are going to use a number from the UK (thanks Adil!) and get a confirmation via SMS:

Send PIN (Zapier)

Now press the Send PIN button to continue. In the meantime, check your mobile phone for a message from Zapier with the PIN number. Once you get that, type it in the related text box.

Send PIN back (Zapier)

Press the Yes, Continue button to complete the process and go to the next screen:

SMS Zapier OK

Clink on the CONTINUE button for the SMS text configuration.

Configure SMS - Zapier

Select Random from the From Number list and Your Kafka broker is down! for the message and press the CONTINUE button. After that press the TEST & CONTINUE button, which will make Zapier send a text message to your mobile.

SMS Test and Continue (Zapier)

You are done creating your Zap. The last action you should perform is turning on that Zap by pressing on the relevant switch that appears at the end of your screen and at the upper right corner. After that, press the DONE EDITING click and select GO TO MY ZAPS.

Your newly created Zap should appear on the Zapier list. Press the down sign on the right side of your Zap and select Task History from the menu that appears.

Task History SMS (Zapier)

Keep that screen open for easy reference. We are done with Zapier – we will now continue with Lenses.

Creating a Lenses Connection

You will use the Zapier webhook to create a new Lenses Connection from Lenses UI. Go to the Lenses ADMIN screen and select Connections. Then click on + New Connection where you should select the Webhook kind of connection.

Based on the address (https://hooks.zapier.com/hooks/catch/6433676/oiooeim/), of the Zapier webhook, you should type hooks.zapier.com on the Host box and turn on the Use HTTPS switch.

Please make sure that you do not put HTTP or HTTPS in front of the value of the Host box. The name of the connection can be whatever you want provided that is an alphanumeric string up to 127 characters long that does not contain any special symbol or whitespaces. For this tutorial, the name of the connection is zapier-sms.

Click on the Save Connection box and you are done. The next screen will present summary information about your connection.

Zapier SMS Lenses connecti

Creating a Lenses Channel

Now select Channels, which is under the ALERTS group, and then click on New Alert Channel. The type of the new channel will be Webhook, so click on the relevant button.

In the screen that follows, you will need to fill in the required information, including the Request path of the channel, which you will discover from the Zapier webhook address – in this case the value will be /hooks/catch/6433676/oiooeim/. The configuration of the channel appears on the next screenshot.

Zapier SMS channel

After that press the Create Alert Channel button and you are done! The name of the new channel is zapier-channel-sms.

Using a Channel for getting Alerts

The last step is about connecting that channel to the desired alert rules – in this case the rule of interest is Kafka Broker is down. For that, you should go the Rules screen and edit the Kafka Broker is down Rule as follows:

SMS Alert Rule

That is the last step for connecting Zapier with Lenses.

Note that you are allowed to associate a channel with multiple rules but this might be confusing sometimes.

Seeing the results

If a Kafka broker goes down or becomes unreachable, Lenses will use the Zapier webhook as a trigger for sending an SMS message. The same will happen when a Kafka broker goes up again!

Zapier SMS messages

Additionally, you can visit the Task History screen of your Zap to see the number of SMS messages that have been sent for that particular Zap.

SMS Task History

Next steps

Now that you know how to send SMS alerts from Lenses, you should begin designing your strategy in order to get notified when something happens to your Kafka infrastructure.

Useful links

Ready to get started with Lenses?

Download free version