Jsonnet is the perfect solution for real-time monitoring with Ceph dashboards
In the world of monitoring large-scale distributed systems, such as Ceph, having comprehensive and insightful dashboards is crucial for understanding the health and performance of the system.
One powerful tool used to generate these dashboards is Jsonnet, a superset of JSON that allows a flexible and dynamic dashboard creation by simplifying the JSON data.
In this blog post, I will show you how Jsonnet is used to build Ceph’s dashboards for monitoring.
Understanding Ceph Monitoring
Before we delve into the dashboard generation process, let’s first gain a basic understanding of Ceph monitoring. Ceph is a popular distributed storage system that provides reliable and scalable storage for various applications. To monitor the health and performance of a Ceph cluster, metrics such as cluster status, disk usage, network throughput, and latency need to be collected and visualized.
To monitor Ceph clusters, we use Grafana dashboards. They can be created manually, but since we need to set a lot of parameters we can consolidate them by using a JSON file that contains the following:
- The JSON file begins with metadata that includes information about the dashboard, such as its title, description, and version. This metadata provides context and identification for the dashboard.
- Panels are the building blocks of a Grafana dashboard. Each panel represents a visual element on the dashboard, such as a graph, table, or singlestat (a single value visualization). The JSON file defines an array of panel objects, each with its own properties like title, type, and configuration specific to the visualization type.
- Grafana allows you to connect to various data sources to fetch and display metrics data. The JSON file contains information about the data sources used by the panels. This includes details such as the data source type (e.g., Prometheus, InfluxDB), connection settings, and query configurations.
- Variables provide a way to make dashboards more interactive and dynamic. They allow users to select different options or filter data based on certain criteria. The JSON file defines variables and their properties, such as name, label, data source, and available values.
- Templating enables parameterization of dashboards by substituting variables into queries and other configurations. The JSON file includes settings for templating, specifying the variables to be used, their query sources, and how they should be displayed.
- Grafana allows users to adjust the time range for which data is displayed on the dashboard. The JSON file includes properties that define the default or preset time range options available to the user, such as “Last 1 hour,” “Last 7 days,” or custom ranges.
- Annotations provide a way to mark specific events or points of interest on the dashboard. The JSON file can contain annotations configuration, including data sources, query settings, and appearance options.
Style and Layout
- The JSON file may include properties for defining the overall style and layout of the dashboard, such as the theme, color scheme, grid options, and sizing of panels.
Links and Navigation
- Grafana allows you to add navigation links to external resources or other dashboards. The JSON file can contain properties that define these links and their associated labels.
What is Jsonnet
Jsonnet is a domain-specific language designed for generating JSON data. It provides a concise and flexible way to define complex JSON structures, making it an ideal choice for dashboard data.
Jsonnet extends JSON, adding variables, conditionals, functions, reusable components, and more to minimize duplication in defining configuration files. It’s got formatters, linters, and editor integrations.
According to the website,
The name Jsonnet is a portmanteau of JSON and sonnet, pronounced ‘jay sonnet’.
It was invented in 2014 at Google, and embodies years of experience configuring complex IT systems. Now, Jsonnet available to everyone. Thanks, Google.
Raise your hand if you just learned a new vocabulary word. (Portmanteau, no?)
By the way, credit for these examples goes to whoever wrote the Jsonnet doc. For this part, we will lean on their work. Kudos.
How it works
Let’s start with a simple example. We can declare a JSON file using an expressions and an object reference.
We can do the same thing using a function instead.
Jsonnet turns this into:
Look familiar? Jsonnet is pretty flexible, giving you lots of options.
Now let’s turn up the complexity a little. We can define some variables for reuse throughout the configuration. Also, Jsonnet can write to many file formats, including
The following example names three output files to generate.
That Jsonnet file produces:
That’s enough to understand Ceph monitoring and Jsonnet in general. Let’s get a preview of what’s coming next.
Part II and Part III, coming soon
In Part II, we will talk about Grafana and its amazing graphing capabilities. Grafana has an impressive set of Jsonnet libraries that support the builder pattern, making it easy to string together widgets and compose beautiful dashboards. So we will dive a little deeper into that.
Then we will wrap up this series by showing how all of this comes together in the latest code for the Ceph dashboard. That will be in Part III.