Skip to content
vedavidvedavidvedavid docs

PromQL patterns

A phone panel is one number or one small list. These are the queries that survive that constraint. Copy them and swap the job label.

Error budget burn

Section titled “Error budget burn”

The single most useful stat panel on a pager. Fast burn over five minutes, compared against your objective.

promql
  sum(rate(http_requests_total{job="gateway-edge",code=~"5.."}[5m]))
/ sum(rate(http_requests_total{job="gateway-edge"}[5m]))

Latency quantiles, three at once

Section titled “Latency quantiles, three at once”

Use the quantiles panel kind and one query per quantile. p50 next to p95 tells you whether it is everyone or the tail.

promql
histogram_quantile(0.95,
  sum by (le) (rate(http_request_duration_seconds_bucket{job="gateway-edge"}[5m]))
) * 1000

The pod that is actually on fire

Section titled “The pod that is actually on fire”

A list panel with topk. Five rows is the most a phone should ask you to read.

promql
topk(5,
  sum by (pod) (rate(container_cpu_usage_seconds_total{namespace="prod"}[5m]))
)

Saturation against the limit, not the box

Section titled “Saturation against the limit, not the box”

Absolute memory is meaningless on a phone. A ratio against the limit is not.

promql
max by (pod) (
    container_memory_working_set_bytes{namespace="prod"}
  / on(pod) group_left kube_pod_container_resource_limits{resource="memory"}
)