In companieshouse/DARr: Prototype RAP package for Companies House

Appendix D: Prometheus & Grafana {-}

Prometheus {-}

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open source project and maintained independently of any company. To emphasize this, and to clarify the project's governance structure, Prometheus joined the Cloud Native Computing Foundation in 2016 as the second hosted project, after Kubernetes.

Prometheus's main features are:

• a multi-dimensional data model with time series data identified by metric name and key/value pairs
• PromQL, a flexible query language to leverage this dimensionality
• no reliance on distributed storage; single server nodes are autonomous
• time series collection happens via a pull model over HTTP
• pushing time series is supported via an intermediary gateway
• targets are discovered via service discovery or static configuration
• multiple modes of graphing and dashboarding support

What are metrics {-}

In layperson terms, metrics are numeric measurements. Time series means that changes are recorded over time. What users want to measure differs from application to application. For a web server it might be request times, for a database it might be number of active connections or number of active queries etc.

Metrics play an important role in understanding why your application is working in a certain way. Let's assume you are running a web application and find that the application is slow. You will need some information to find out what is happening with your application. For example the application can become slow when the number of requests are high. If you have the request count metric you can spot the reason and increase the number of servers to handle the load.

Components {-}

The Prometheus ecosystem consists of multiple components, many of which are optional:

• the main Prometheus server which scrapes and stores time series data
• client libraries for instrumenting application code
• a push gateway for supporting short-lived jobs
• special-purpose exporters for services like HAProxy, StatsD, Graphite, etc.
• an alertmanager to handle alerts
• various support tools
• Most Prometheus components are written in Go, making them easy to build and deploy as static binaries.

Architecture {-}

This diagram illustrates the architecture of Prometheus and some of its ecosystem components:

knitr::include_graphics("./images/architecture.png")

Prometheus scrapes metrics from instrumented jobs, either directly or via an intermediary push gateway for short-lived jobs. It stores all scraped samples locally and runs rules over this data to either aggregate and record new time series from existing data or generate alerts. Grafana or other API consumers can be used to visualize the collected data.

When does it fit? {-}

Prometheus works well for recording any purely numeric time series. It fits both machine-centric monitoring as well as monitoring of highly dynamic service-oriented architectures. In a world of microservices, its support for multi-dimensional data collection and querying is a particular strength.

Prometheus is designed for reliability, to be the system you go to during an outage to allow you to quickly diagnose problems. Each Prometheus server is standalone, not depending on network storage or other remote services. You can rely on it when other parts of your infrastructure are broken, and you do not need to setup extensive infrastructure to use it.

When does it not fit? {-}

Prometheus values reliability. You can always view what statistics are available about your system, even under failure conditions. If you need 100% accuracy, such as for per-request billing, Prometheus is not a good choice as the collected data will likely not be detailed and complete enough. In such a case you would be best off using some other system to collect and analyze the data for billing, and Prometheus for the rest of your monitoring.

Grafana {-}

Grafana is a multi-platform open source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources. A licensed Grafana Enterprise version with additional capabilities is also available as a self-hosted installation or an account on the Grafana Labs cloud service. It is expandable through a plug-in system. End users can create complex monitoring dashboards using interactive query builders. Grafana is divided into a front end and back end, written in TypeScript and Go, respectively.

As a visualization tool, Grafana is a popular component in monitoring stacks, often used in combination with time series databases such as InfluxDB, Prometheus and Graphite; monitoring platforms such as Sensu Icinga, Checkmk, Zabbix, Netdata, and PRTG; SIEMs such as Elasticsearch and Splunk; and other data sources. The Grafana user interface was originally based on version 3 of Kibana.

Grafana doesn’t require you to ingest data to a backend store or vendor database. Instead, Grafana takes a unique approach to providing a “single-pane-of-glass” by unifying your existing data, wherever it lives.

Aggregating time series {-}

Depending on what you’re measuring, the data can vary greatly. What if you wanted to compare periods longer than the interval between measurements? If you’d measure the temperature once every hour, you’d end up with 24 data points per day. To compare the temperature in August over the years, you’d have to combine the 31 times 24 data points into one.

Combining a collection of measurements is called aggregation. There are several ways to aggregate time series data. Here are some common ones:

• Average returns the sum of all values divided by the total number of values.
• Min and Max return the smallest and largest value in the collection.
• Sum returns the sum of all values in the collection.
• Count returns the number of values in the collection.

For example, by aggregating the data in a month, you can determine that August 2017 was, on average, warmer than the year before. Instead, to see which month had the highest temperature, you’d compare the maximum temperature for each month.

How you choose to aggregate your time series data is an important decision and depends on the story you want to tell with your data. It’s common to use different aggregations to visualize the same time series data in different ways.

knitr::include_graphics("./images/agg.png")

companieshouse/DARr documentation built on Oct. 22, 2022, 8:26 p.m.