Monitoring Docker with Prometheus

Prometheus makes it very easy to monitor Docker containers. It is done in a completely non-intrusive way. In this blog post, we will use an exporter called cAdvisor to monitor docker containers. cAdvisor(short for container advisor) is a google project written in go lang and is quite actively maintained.

The blog entry will have the following sections

• Setting up cAdvisor on a Linux instance
• Configure Prometheus server
• Query using PromQL
• Grafana dashboard for docker

This blog entry assumes that there is already a Prometheus server available. To learn more about how to install a Prometheus server please refer to this blog entry.

Setting up cAdvisor

cAdvisor binaries can be downloaded from Github. For the use of this blog entry, we are using the latest cAdvisor 0.36.0. cAdvisor along with node_exporter makes a perfect tool for monitoring Docker containers. In case, you want to quickly get started with node_exporter read this blog post.

You can download the cAdvisor onto your linux box using the following command

Once downloaded – cAdvisor should be started using elevated privileges user. Now the reason is that it looks the docker data dir and also monitors the docker process which itself in root user.

Note: Do not use the docker image to launch cAdvisor containers as stopping and removing your existing containers causes all sorts of issues.

Once downloaded, cAdvisor can be easily started using the following command

Once cAdvisor is running – the metrics are available on the following http://<host>:9600/metrics. See Below

But interestingly, cAdvisor also gives you a standalone UI as well. The URL for that is http://<host>:9600/containers. See Below

Configure Prometheus server

For Prometheus server to scrape metrics from the above Linux server additional configuration needs to be added. These are put in prometheus.yml. See below

If you want to run multiple targets just add a comma and add another.

It’s time to start the Prometheus server. If you need more information on how to install/run/configure Prometheus server please refer to this blog entry.

You can check if Prometheus server is able to scrape the metrics is by navigating to Prometheus UI on http://<host>:<port>/graph

This would bring up something similar as below

Query using PromQL

To check if Prometheus server is able to scrape metrics from cAdvisor goto Prometheus UI. Type in cadvisor_version_info and it should return a result. See below.

If there are two cAdvisor exporters running you will see two rows for this metric. Well our Prometheus server is now able to scrape the metrics being served up cAdvisor. Take a breather and explore the metrics there is a huge list.

Grafana dashboard for docker

So our Prometheus server is now able to scrape docker metrics. Its time to import a grafana dashboard for docker to get us going on Docker metrics. For the purpose of this blog entry, I am going to import a dashboard on this link.

To import a grafana dashboard follow these steps

Step 1 – Press the + button as shown below

Step 2 – You can import by typing the id assigned by grafana website to the dashboard or directly paste the json. I have decided to just type in the id. See Below

Step 3 -Select the data source and folder name. Press import.

The dashboard is ready!

This works across multiple nodes. You can now add/change/remove charts to suit your requirements. You can explore various individual metrics and come up with something new!

This brings us to the end of this entry. Hope you have found this entry useful. If you like it – share it. Maybe leave a comment! 🙂