Healthy server, waiting users: why average speed can hide the truth
Tail latency, queues and real user response time: what to measure beyond averages and how to avoid buying capacity for the wrong problem.
Conceptual source: Monitoring in Google SRE Workbook. It explains limitations of averages and the value of latency distributions. The example and investigation plan below are independent proposals, not a diagnosis of this website or a product benchmark.
A user reports a slow page while the dashboard looks normal. Neither must be wrong. Most requests may be fast while the user encounters a rare slow path that the average obscures.
Move from a number to a distribution
Averages help with some trends but do not describe the shape of experience. I would also examine percentiles, request counts and errors. p95 marks a boundary covering most measured observations; it is neither a promise for every person nor a definite worst case.
| Metric | What it helps reveal | What it does not prove |
|---|---|---|
| Median | Typical experience in the measured set | Health of uncommon paths |
| p95 | The slower part of the distribution | Absence of much slower requests |
| Errors and abandonment | Work that did not complete | Root cause without investigation |
| Queue waiting time | Waiting versus execution | That adding CPU must solve it |
The measured population matters. Excluding timed-out or failed requests can make a service look faster as experience worsens. High percentiles can also be unstable with small samples.
A queue is not processor utilization
In a hypothetical scenario, an application waits for an available database connection. Host CPU use can remain low while users wait. Queues may sit at the proxy, connection pool, background worker or external service. A larger server does not necessarily remove them.
Longer response deadlines are not a universal remedy either: they may simply permit longer queues. Arrival rate, completion capacity and overload decisions should be designed together.
Follow a request from the user’s perspective
- Select an important journey such as search or opening a report.
- Measure user elapsed time separately from application time.
- Correlate dependency waits and execution with a request identifier.
- Compare one bounded change under similar load, including errors and result quality.
OpenTelemetry’s trace concept is a useful starting point. Tracing alone does not represent every request: sampling and uninstrumented stages create blind spots. Sensitive information should not be unnecessarily stored in events.
Avoid averaging percentiles
Averaging server percentiles generally does not produce the percentile of all requests. Aggregation must match the data and measurement tool. Know which requests, time window and exclusions produced the chart.
Management reports should distinguish important journeys too. Mixing a lightweight public page and a costly report into one bucket can make the result unsuitable for decisions about either.
Practical questions
Does every slowdown require more RAM or CPU?
No. Capacity helps when the limiting resource is known. Connection settings, query improvements or removing duplicated work may matter more; measure the result.
Is a test from one computer enough?
It helps investigate a path, but does not establish experience across networks and devices. Combine external tests, user data and server observations.
When is troubleshooting complete?
When the same journey meets predefined improvement criteria under comparable conditions without shifting errors or quality problems elsewhere. A green chart is insufficient.
Related reading: a live server versus a healthy service.
Sources checked: November 2, 2026. The scenario and steps are original analysis, not a fixed improvement promise or tool endorsement.
Continue reading
All posts →
The ownerless service account: a small access path easily forgotten
Give each service account a short card: purpose, owner, permissions, secure credential reference and review date, without putting passwords in public documents.
Not ready or actually stuck? One health check cannot make every decision
Separate readiness for traffic from application liveness so a temporary dependency fault does not become a restart loop.
Investigate 502 across three boundaries, before restarting everything
Separate client, proxy and upstream faults using time and request identifiers, with a short investigation path that avoids premature conclusions.