Prometheus : Monitor docker services with grafana

Here is a little tutorial to setup Prometheus monitoring for a local docker daemon and host metrics with docker-compose and :

• prometheus node exporter for system metrics
• cadvisor for docker metrics

First create a prometheus.yml config file like this :

global:
alerting:
  alertmanagers:
    - static_configs:
      - targets:
#       - alertmanager:9093
rule_files:
 # - "first_rules.yml"
 # - "second_rules.yml"
scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']
    file_sd_configs:
      - files:
        - '/etc/prometheus/file_sd_configs.yml'

And a file service discovery config file_sd_configs.yml:

- targets:
  - grafana:3000
  - prom-node-exporter:9100
  - cadvisor:8080

Now it’s time to start all services with a docker-compose file.

I’m still using the great traefik for reverse proxy and ssl let’s encrypt certificate generation.

So here is my docker-compose :

version: '2'
services:
   traefik:
    image: traefik
    hostname: traefik
    container_name: traefik
    command: -c /etc/traefik/traefik.toml
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/data/traefik/traefik.toml:/etc/traefik/traefik.toml"
      - "/data/traefik/acme.json:/etc/traefik/acme.json"
      - "/data/traefik/ssl:/etc/traefik/ssl"
      - "/var/run/docker.sock:/var/run/docker.sock"

  prometheus:
    image: prom/prometheus
    hostname: prometheus
    container_name: prometheus
    volumes:
      - "/data/docker/volumes/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml"
       - "/data/docker/volumes/prometheus/file_sd_configs.yml:/etc/prometheus/file_sd_configs.yml"
     labels:
      - "traefik.frontend.rule=Host:prometheus.aperogeek.fr"
 
  prom_node_exporter:
    image: prom/node-exporter
    hostname: prom-node-exporter
    container_name: prom-node-exporter
    labels:
      - "traefik.enable=false"
 
  cadvisor:
    image: google/cadvisor:latest
    hostname: cadvisor
    container_name: cadvisor
    volumes:
      - "/:/rootfs:ro"
      - "/var/run:/var/run:rw"
      - "/sys:/sys:ro"
      - "/var/lib/docker/:/var/lib/docker:ro"
    labels:
      - "traefik.enable=false"
 
  grafana:
    hostname: grafana
    container_name: grafana
    image: grafana/grafana
    environment:
      - GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-piechart-panel
    volumes:
      - /data/grafana:/var/lib/grafana
    labels:
      - "traefik.frontend.rule=Host:grafana.aperogeek.fr"

Then I needed to create a prometheus datasource in grafana

And finally I’ve just imported some public grafana dashboards. 

And voila 🙂