SoloMedium6m

The P99 Latency Timeout

#sev1 — SRE dashboard P99 widget timing out

VP Eng: Active incident, dashboard blank. Backend sorts 50M encrypted logs with sorted() to read one percentile — 45s compute, 30s gateway timeout.

The trap: P99 is a selection problem, not a sorting problem. Full sort is O(N log N).

Your task

Fix `MetricsAggregator` in `latency_metrics.py`: implement `quick_select()` (partition + recurse one side) and use it in `get_p99_latency()`.

Book: sort then index · Prod: Quick Select O(N) — discard half each step.

Skills you'll prove

algorithmspythonquick-selectdivide-and-conquertime-complexitypercentilessrebackendbook-vs-prod

Interview Tip

When asked about percentiles or latency metrics, say: "You rarely need to sort the entire dataset to find P99. I debugged an SRE dashboard that timed out because someone used sorted() on 50 million log entries — O(N log N) with expensive comparisons. The fix was Quick Select: partition around a pivot and recurse only into the half containing the target index, dropping average complexity to O(N). In production at scale you often use t-digest or streaming approximations, but the key insight is the same — don't process data you can discard."

Browse freely — sign in when you're ready to open a ticket and prove your skills.