Grafana

Grafana is an open source, feature rich metrics dashboard and graph editor for Graphite, InfluxDB & OpenTSDB. OpenStack runs Graphite which stores all the metrics related to Nodepool and Zuul (to name a few).

Overview

Apache is configured as a reverse proxy to Grafana running in a container, listening on port 3000.

Local Development

To develop dashboards, you can run the OpenDev Grafana container. Firstly, get setup:

$ cd <work dir>
$ mkdir secrets
$ echo "password" > secrets/admin_password
$ echo "admin" > secrets/admin_user
$ echo "key" > secrets/secret_key

$ git clone https://opendev.org/openstack/project-config

Then run the container with the following options:

$ cd <work dir>
$ sudo podman run \
  -p 3000:3000 \
  -v ./secrets:/etc/grafana/secrets \
  -v ./project-config:/opt/project-config \
  -e GF_AUTH_ANONYMOUS_ENABLED=true \
  -e GF_USERS_ALLOW_SIGN_UP=false \
  -e GF_SECURITY_ADMIN_PASSWORD__FILE=/etc/grafana/secrets/admin_password \
  -e GF_SECURITY_ADMIN_USER__FILE=/etc/grafana/secrets/admin_user \
  -e GF_SECURITY_SECRET_KEY__FILE=/etc/grafana/secrets/secret_key \
 docker.io/opendevorg/grafana

At this point, Grafana will be running and listening on port 3000. You can log into as admin with password (or using your secrets above).

This is unconfigured and does not yet talk to the OpenDev Graphite instance. The dashboard definitions are kept in project-config/grafana. To load them exec the update-grafana script in the container (i.e. podman exec <running-container> update-grafana). That will run grafyaml and load in the updated dashboards (note it relies on things being mapped as above). To work on dashboards, update the yaml files in project-config and re-run update-grafana, then reload them in the Grafana UI.

Alternatively, you can use the Grafana editor to make the dashboards, and then under “Dashboard Settings” (gear icon) select “JSON Model” and commit that (it seems you have to cut-and-paste, there isn’t currently a way to export the JSON as such).