One Customer Hijacked the GPU
P1 — TENANT ADMISSION MONOPOLY
On-call: An enterprise customer queued 4,000 background summaries. Aggregate throughput looks healthy, but interactive users wait behind the bulk queue for their first token.
Your job: Fix only `select_next_request()` in `scheduler.py`. Apply the supplied weighted class cycle, rotate eligible tenants, and respect aggregate tenant concurrency caps without leaving usable capacity idle.
Investigate: Run python fairness_monitor.py. A deterministic CPU replay shows chat first-token delay, background completions and cap violations. The trap is replacing bulk-first FIFO with absolute interactive priority, which starves background work under sustained load.
Proof: Complete all 24 chat requests within the replay's first-token target while background work progresses and caps hold. Run python -m pytest -v, then open a PR against main.
Book: Process requests in arrival order. Prod: Shared infrastructure must protect different service requirements without starving anyone.
This is an authored gateway admission policy, not a claim that an engine priority flag provides tenant fairness. Simulated ticks isolate admission delay; no GPU or model download is needed.
Skills you'll prove
Interview Tip
I protect interactive traffic with weighted admission while preserving background opportunities, then rotate tenants within each class. Per-tenant concurrency caps stop one customer from occupying the entire service. Empty or blocked classes yield their share so capacity stays useful. I verify latency and background progress together because aggregate throughput can hide starvation.