How to create an Alert Channel with Prometheus Alertmanager

Introduction

Lenses 3.1 is finally here with lots of improvements including a better and faster way to send alerts to alert management solutions through different Channels. In this blog post you are going to see how to setup Lenses 3.1 to send alerts to Prometheus Alertmanager.

Pre-requisites

In order to be able to follow the steps of this blog, you will need the following:

• Lenses 3.1 (or a newer version when available) up and running. You can use our free Box instance if necessary.

• Prometheus Alertmanager up and running

• A working network connection between these two machines

The Implementation

The process requires performing the following three steps:

• Creating a Connection

• Creating a Channel

• Associating that channel to one or more existing alerts

Creating a Connection

This step is about pointing Lenses to the desired Alertmanager instance using a connection template. First, you should go to the ADMIN panel and select Connections. Then, you should click on the New Connection button that will get you to the screen that allows you to select the service that you are going to use, which in this case is Prometheus Alertmanager.

The next screenshot shows the available templates at the time of writing this.

Click on PrometheusAlertmanager box in order to give the details of the connection. This is illustrated in the next screenshot.

Note that in this case the hostname of the machine that runs Alertmanager is alertmanager. You should change that to match your configuration.

Now that you have given the required information, press on the Save Connection button to create the connection and you are done. Your connection will be now listed on the Connections tab of the ADMIN panel.

Creating a Channel

You should now go to the ADMIN panel and select the Channels tab. Then, you should click on the New Alert Channel button in order to create your channel. First, you will need to select the Channel Type from a list that looks as follows:

Click on the button for Prometheus Alertmanager. After that, you will be asked to fill in some information.

You can put any name you want on the Channel text box. However, on the Select connection box you should select the appropriate connection from the presented list, which is this case will be AlertManagerConnection. The Source box should have a descriptive name for your channel and the Generator URL box should have the URL of your Lenses server.

Click on the Create Alert Channel button and you are done. The only thing left is associating your new channel with one or more alerts.

Associating a channel with an existing alert

The last step requires that you associate a channel with one or more existing alerts. Go to the ADMIN panel and select Rules from ALERTS. Press the Edit icon on the right of the Infrastructure rules that you want to associate with one or more Channels.

Note that you should click on the empty box on the left of a channel name in order to associate that channel with a rule.

The configuration used in this blog post is presented in the next screenshot.

You should now wait for an event to happen in order for Lenses to send an alert to Alertmanager.

Visiting the Alertmanager UI

Prometheus Alertmanager usually uses port number 9093, which means that we should visit the http://alertmanager:9093 URL to see its UI, which will look similar to the following:

If a Kafka broker becomes unavailable or if the Zookeeper server is down, you will get the following kind of output on Alertmanager.

The previous output verifies that Lenses 3.1 can successfully communication with the Prometheus Alertmanager server. Additionally, it shows that there exist two CRITICAL alerts. The first alert is for Zookeeper whereas the second alert is for a Kafka broker.

Next Steps

Now that you know how easy it is to set up alerts with Lenses 3.1, you should start using that handy feature as soon as possible.

Other Links

• Lenses Box 5 min tour

• Prometheus Alertmanager