Docs Problems & alerting
SLOs and error budgets
Define service level objectives for availability, errors or latency, track the error budget over a rolling window, and get alerted on multi-window burn rates.
A service level objective is a promise in numbers. "99.9% of requests answer without an error over 30 days" is one. Obscenter tracks how much of the promise is used up, the error budget, and how fast it is being spent, the burn rate. Find the page under Launchpad → Frontend → SLOs.
Three kinds of objective
| Objective | Good events | Out of |
|---|---|---|
| A synthetic check stays available | Runs that passed | All runs |
| A service answers without errors | Requests that succeeded | All requests |
| A service answers fast enough | Requests at or under the threshold | All requests |
Press + New objective, choose the kind, and name the check or the service. For the latency kind, set Threshold ms, which is 500 by default.
- Target %: 99.9 by default. It must be at least 90 and below 100. 100% is not a target, it is a budget of zero.
- Window (days): 30 by default, from 1 to 90.
The error budget
A 99.9% target over 30 days allows 0.1% bad events. For time-based thinking, that is about 43 minutes of full outage. The page shows for each objective:
- the attainment so far;
- how much of the budget is spent;
- how many bad calls or runs are left;
- the burn rate, now and over six hours.
The budget bar turns amber at 60% spent and red when it is gone.
Burn rate
A burn rate of 1× spends the budget exactly over the window. 14× would spend a 30-day budget in about two days. The rate is measured over two windows, one hour and six hours. The short window reacts. The long one makes sure a short spike is not mistaken for a trend.
| State | When | What happens |
|---|---|---|
| ok | Otherwise | Nothing |
| warning | 1 h ≥ 6× and 6 h ≥ 3× | A badge on the page, no ticket |
| burning | 1 h ≥ 14.4× and 6 h ≥ 6× | A serious ticket: the error budget is burning fast |
| breached | Attainment below the target | A critical ticket: the objective has been breached |
SLO tickets go through the same policies and channels as every other ticket.
Choosing objectives
- Start from what users notice. A checkout that fails matters more than a health endpoint.
- For a public page, an availability objective on a synthetic check measures it even when nobody is visiting.
- For an API, pair an error objective with a latency objective at a threshold your users would call slow.
- Set the target from what the service has actually done, then tighten it. A target that is always breached teaches people to ignore it.
Who sees them
An objective is visible to whoever can see the check or service it watches. Viewers can read objectives but cannot create or delete them. The AI assistant can read error budgets, if you want to ask which objectives are in trouble.