The Sourcegraph monitoring generator uses Container definitions to generate integrations with Sourcegraph's monitoring architecture.
It also aims to help codify guidelines defined in the Sourcegraph monitoring pillars.
This page primarily documents the generator's current capabilities - in other words, and what you get for free by declaring Sourcegraph service monitoring in this package - as well as how to make changes to the generator itself.
To learn about how to find, add, and use monitoring, see the Sourcegraph monitoring developer guide.
From this directory:
go generate ./...The generator automatically creates documentation from monitoring definitions, such as alert solutions references, that customers and engineers can reference.
Links to generated documentation can be provided in our other generated integrations - for example, Slack alerts will provide a link to the appropriate alert solutions entry.
The generator automatically generates and ships dashboards from monitoring definitions within the Sourcegraph Grafana distribution.
It also takes care of the following:
- Graphs within rows are sized appropriately
- Alerts visualization through the
ObservableAlertDefinitionAPI:- Overview graphs for alerts (both Sourcegraph-wide and per-service)
- Threshold lines for alerts of all levels are rendered in graphs
- Formatting of units, labels, and more (using either the defaults, or the
ObservablePanelOptionsAPI) - Maintaining a uniform look and feel across all dashboards
Links to generated documentation can be provided in our other generated integrations - for example, Slack alerts will provide a link to the appropriate service's dashboard.
The generator automatically generates and ships Prometheus recording rules and alerts within the Sourcegraph Prometheus distribution. This includes the alert_count recording rules and native Prometheus alerts, all with appropriate and consistent labels.
Generated Prometheus recording rules are leveraged by the Grafana integration.
The generator's Prometheus integration is a critical part of the Sourcegraph's alerting capabilities, which handles alert routing by level and formatting of alert messages to include links to documentation and dashboards. Learn more about using Sourcegraph alerting in the alerting documentation.
At Sourcegraph, routing based on team ownership (as defined by ObservableOwner) is used to route customer support requests and on-call events through OpsGenie.
The Sourcegraph monitoring generator consists of three components:
- The main program - this is the primary entrypoint to the generator.
- Definitions, defined in the top-level
monitoring/definitionspackage. This is where the all service monitoring definitions lives. If you are editing monitoring, this is probably where you want to look - see the Sourcegraph monitoring developer guide. - Generator, defined in the nested
monitoring/monitoringpackage package. This is where the API for service monitoring definitions is defined, as well as the generator code.
All features and capabilities for developed for the generator should align with the Sourcegraph monitoring pillars.