SLO and Error Budget Guide: Set Targets, Calculate Burn Rate, and Plan Reliability Work

Overview

An SLO is a target for a user-visible aspect of service performance over a defined period. For example, a team might target successful request availability, acceptable response latency, or timely data processing. The SLO is supported by an SLI: the measurement and calculation that show whether the target is being met.

The error budget is the amount of unreliability permitted by the SLO. If the target is 99.9% successful requests during a 30-day window, the allowed failure fraction is 0.1%. With 43,200 minutes in a 30-day period, a time-based availability budget would be 43.2 minutes. For request-based availability, the same target allows 1 failed or unacceptable request for every 1,000 total requests, subject to the service's measurement rules.

Error budgets are most useful when they connect observability data to decisions. A healthy budget can support planned delivery work when other safeguards are in place. A rapidly shrinking or exhausted budget should prompt investigation, safer rollout methods, remediation, or a temporary change in release risk. The budget is not a promise that every incident is acceptable; it is a planning mechanism for balancing reliability and change.

Choose the measurement boundary carefully. A customer-facing API may need a request-based SLI, while a batch pipeline may need freshness or completion-time indicators. A Kubernetes health check alone is usually too narrow to represent the user experience. Pair infrastructure signals with service-level outcomes where possible. For broader context, see the Prometheus and Grafana monitoring guide and the Kubernetes troubleshooting guide.

How to estimate

Use the following workflow to create an SLO and estimate its remaining budget.

1. Define the user journey. Identify the action whose reliability matters: placing an order, loading a dashboard, authenticating, receiving an event, or completing a scheduled job.
2. Select one or more SLIs. Use a numerator and denominator that can be measured consistently. For request success, the numerator may be successful requests and the denominator all eligible requests. For latency, the numerator may be requests below a chosen threshold and the denominator all eligible requests.
3. Set the target and window. Express the target as a percentage or threshold and state whether it is evaluated over a rolling window or a fixed calendar period. Make exclusions explicit, such as intentional test traffic or requests rejected by a documented client-side condition.
4. Calculate the allowed bad-event fraction. Subtract the target from 1. For a 99.9% target, the allowed fraction is 0.001, or 0.1%.
5. Measure the observed bad-event fraction. Divide bad events by total eligible events for the same window. For latency, count requests that exceed the threshold as bad events.
6. Calculate burn rate. Divide the observed bad-event fraction by the allowed bad-event fraction. A burn rate of 1 means the budget is being consumed at the average rate that would exhaust it by the end of the window. A burn rate above 1 indicates faster consumption.
7. Define an operational response. Decide what happens at warning, investigation, and exhaustion levels. The response should identify an owner and a concrete action rather than simply create another dashboard.

The basic formulas are:

• SLI: good events ÷ eligible events
• Error budget fraction: 1 − SLO target
• Observed bad-event fraction: bad events ÷ eligible events
• Burn rate: observed bad-event fraction ÷ error budget fraction
• Remaining budget: allowed bad events − observed bad events

Burn rate is a rate of budget consumption, not a complete prediction. Traffic patterns, incident clustering, retries, sampling, and changes in eligibility can make a short-term rate misleading. Use multiple windows and inspect the underlying events before taking a high-impact action.

Inputs and assumptions

A reliable SLO calculation depends more on measurement design than on arithmetic. Record these inputs with every SLO:

• Service scope: name the service, endpoint group, workflow, or customer segment covered.
• Eligible events: define which requests, jobs, or transactions enter the denominator.
• Bad-event rules: document status codes, timeout limits, validation failures, freshness limits, and latency thresholds that count against the budget.
• Window: choose a rolling or calendar window and use the same interpretation in dashboards, alerts, and reviews.
• Data source: identify the metrics, traces, logs, or event records used to calculate the SLI. Avoid relying on a metric that can silently disappear during an outage.
• Aggregation: state whether the result is calculated globally, by region, by tenant, by endpoint, or by another dimension. A global average can hide a serious failure affecting a small but important group.
• Target rationale: explain why the target is appropriate for the user journey and business impact. Do not select extra decimal places merely because a monitoring system makes them available.
• Ownership: assign responsibility for the dashboard, alert rule, SLO review, and remediation work.

For latency SLOs, percentiles can be useful, but define the population and threshold precisely. “Fast” is not an SLI. “At least 99% of eligible requests complete within 500 milliseconds” is specific enough to calculate, test, and review. For asynchronous systems, consider freshness, queue age, successful completion, and end-to-end delivery rather than only producer or consumer health.

Alert windows should match the decision being protected. A short window can detect a severe incident quickly but may be noisy during brief traffic changes. A long window gives more context but can delay response. A practical design often combines a short window for urgent symptoms with a longer window for sustained budget consumption. Tune thresholds against historical service behavior and incident response capacity instead of copying a universal number.

Worked examples

Request availability

Assume an API has a 99.9% SLO over 30 days. Its error budget is 0.1%. During a review period, the service receives 100,000 eligible requests and 250 are classified as failures.

• Observed bad-event fraction: 250 ÷ 100,000 = 0.0025, or 0.25%
• Allowed bad-event fraction: 0.001, or 0.1%
• Burn rate: 0.0025 ÷ 0.001 = 2.5

The service is consuming budget 2.5 times faster than the average rate compatible with the full window. The team should inspect whether the failures are concentrated by deployment version, endpoint, region, dependency, or customer segment. A release should not be paused solely because a formula produced a number; it should be evaluated against the team's documented policy and the evidence behind the SLI.

Latency

Suppose a search service defines an SLO that 99% of eligible queries complete within 750 milliseconds. In a measurement window, 50,000 queries are recorded and 800 exceed the threshold.

• Observed bad-event fraction: 800 ÷ 50,000 = 0.016, or 1.6%
• Allowed bad-event fraction: 1 − 0.99 = 0.01, or 1%
• Burn rate: 0.016 ÷ 0.01 = 1.6

This result indicates budget consumption faster than the target's planned average. Investigation might compare latency by query type, dependency, deployment, and resource saturation. Do not replace the latency SLO with CPU or memory utilization simply because those metrics are easier to collect; infrastructure signals help explain the user-facing result but do not define it.

Error-budget review template

Use this compact template in a weekly or monthly review:

• Service and owner: ______
• SLO and window: ______
• SLI definition and data source: ______
• Budget at window start: ______
• Budget consumed and remaining: ______
• Current and peak burn rate: ______
• Largest contributing incidents or changes: ______
• Planned releases affected: ______
• Reliability action, owner, and due date: ______
• Measurement or policy change required: ______

When to recalculate

Recalculate the SLO and its error budget whenever the service, traffic, user journey, or measurement system changes materially. Review it before launching a new service, splitting an API, changing a dependency, introducing a new region, or moving a workload between platforms. Revisit the calculation after an incident if the SLI failed to represent the actual customer impact or if exclusions concealed meaningful failures.

Recheck alert thresholds when traffic volume changes, seasonality becomes important, or responders report excessive noise or late detection. Validate dashboards and queries after telemetry schema changes, instrumentation updates, sampling changes, metric renames, or migration to a new observability platform. If you use OpenTelemetry or another instrumentation layer, confirm that spans and metrics preserve the attributes needed to calculate the SLI without creating unbounded cardinality.

At each review, take three practical actions: verify the denominator, compare the result with known incidents, and confirm that the error-budget policy still leads to a clear decision. Link budget exhaustion to a named response, such as halting a risky rollout, prioritizing a reliability fix, or scheduling a design review. Track the action to completion in the same workflow used for incidents and change management. Teams that maintain infrastructure as code can also version alert and SLO definitions alongside service configuration; the Terraform best practices guide provides relevant guidance for maintainable infrastructure changes. Finally, document every adjustment so future reviews can distinguish a genuine reliability improvement from a change in measurement.