Simulations

P1 — SHARDING WITHOUT CAPACITY Inference platform: The team spread one model across four GPUs without comparing independent replicas. The model fits, but serving capacity disappoints. Your job: Fix only choose_deployment(candidates, workload, budget) in deployment_selector.py. Qualify memory, GPU count, topology and workload compatibility before ranking latency-compliant fleet goodput. Investigate: Run python deployment_monitor.py. Supplied CPU profiles compare replicas and sharding under the same four-GPU budget. A small model favors replicas, a larger model requires sharding, and a fast-link profile makes sharding win on performance. Proof: Include per-GPU KV, workspace and reserved memory; use TP times replicas for total GPU count; never multiply fleet goodput twice. Return no choice if every candidate is infeasible. Run python -m pytest -v, then open a PR against main. Book: More GPUs make the model faster. Prod: Sharding one instance and running more instances solve different problems. Profiles are authored evidence, not actual GPU benchmarks or universal scaling formulas. No GPU or model download is required.

P1 — SPECULATION REGRESSION On-call: Four-token drafts accelerated repetitive completions. After traffic changed, the policy still selects them for high acceptance, but drafting and verification erase the benefit. Your job: Fix only choose_decode_policy(profile) in decode_policy.py. Compare full cost per committed token, validate profile compatibility and memory, and keep ordinary decoding as a fallback. Investigate: Run python speculation_monitor.py. Supplied CPU profiles show draft-4 winning at 3 ms per committed token on one workload, but losing at 12 ms versus the baseline's 10 on changed traffic—even with 100% acceptance. Proof: Choose supported, target-verified policies from matching workload, concurrency and revision profiles. Count drafting, verification and overhead; never bypass target verification. Run python -m pytest -v, then open a PR against main. Book: Generate several draft tokens at once. Prod: Only useful if committed tokens justify the extra work. These are authored timing records and a toy target-authority boundary, not real GPU benchmarks or a speculative sampling implementation.

P1 — PREFIX ROUTING MISS On-call: Requests repeat a long system prompt, but round-robin keeps missing the worker holding its prefix. Pinning all traffic to that warm worker then creates a queue hotspot. Your job: Fix only choose_worker(request, workers) in router.py. Balance reusable prefill against queue delay, require healthy compatible workers, and reject stale cache credit. Investigate: Run python routing_monitor.py. Compare a warm idle worker, a warm overloaded worker and a repeated-prefix burst using supplied CPU timing profiles. Proof: Warm idle routing estimates 40 ms to first content versus 120 ms cold, but 500 ms of queue reverses that choice. The fixed burst improves prefix reuse while spreading work. Run python -m pytest -v, then open a PR against main. Book: Cache repeated work. Prod: Route to compatible reusable state without overwhelming its worker. These are synthetic profiles, not GPU measurements. Cache fingerprints and compatibility metadata are supplied; the learner repairs routing rather than implementing a tokenizer or cache.

P1 — TOKEN WORKLOAD BLIND SPOT On-call: Larger documents launched. Request rate remains 15 RPS, but queued token work triples and first-token latency climbs. The request-count autoscaler sees no change. Your job: Fix only desired_replicas(snapshot, policy) in autoscaler.py. Use supplied incoming and queued token-work estimates, sustained observations, ready versus starting capacity, cooldown and budget limits. Investigate: Run python autoscaling_monitor.py. The authored CPU replay follows a controller through workload growth, replica warmup and a quieter period. Return desired total capacity, not replicas to add. Proof: Long-document demand raises the target from two to four; a repeated decision counts the two replicas already starting. Scale-down preserves busy replicas and respects cooldown. Run python -m pytest -v, then open a PR against main. Book: Scale with requests per second. Prod: Requests contain different amounts of work, and new capacity takes time. The supplied workload profiles are synthetic. This tests scaling decisions and reconciliation, not real GPU throughput or measured latency recovery.

P1 — TENANT ADMISSION MONOPOLY On-call: An enterprise customer queued 4,000 background summaries. Aggregate throughput looks healthy, but interactive users wait behind the bulk queue for their first token. Your job: Fix only select_next_request() in scheduler.py. Apply the supplied weighted class cycle, rotate eligible tenants, and respect aggregate tenant concurrency caps without leaving usable capacity idle. Investigate: Run python fairness_monitor.py. A deterministic CPU replay shows chat first-token delay, background completions and cap violations. The trap is replacing bulk-first FIFO with absolute interactive priority, which starves background work under sustained load. Proof: Complete all 24 chat requests within the replay's first-token target while background work progresses and caps hold. Run python -m pytest -v, then open a PR against main. Book: Process requests in arrival order. Prod: Shared infrastructure must protect different service requirements without starving anyone. This is an authored gateway admission policy, not a claim that an engine priority flag provides tenant fairness. Simulated ticks isolate admission delay; no GPU or model download is needed.

P1 — FALSE BENCHMARK WIN Inference release: The candidate produces more tokens per second, but customers wait longer for answers. The benchmark counts empty heartbeats as first tokens and hides failed and rejected requests. Your job: Fix only summarize_run(events, policy) in benchmark.py. Measure first content and streaming gaps, report all request outcomes, and calculate SLO-compliant goodput over the fixed observation window. Investigate: Run python benchmark_monitor.py. Four requests reach each configuration; the candidate emits more tokens but gives only one customer a timely complete answer. Use the supplied CPU traces and measurement policy—no GPU or model download required. Proof: The baseline delivers 0.3 good requests/s versus the candidate's 0.1. Failed, rejected and unfinished requests stay visible, so dropping difficult traffic cannot manufacture a qualified win. Run python -m pytest -v, then open a PR against main. Book: Higher throughput means better inference. Prod: Better for which users, at which latency and failure rate? The fixtures are authored traces, not hardware measurements. The exercise teaches measurement correctness and honest release comparison, not real-GPU performance certification.

P1 — GHOST GENERATIONS On-call: Only 12 browsers are connected, but 48 generations still hold serving capacity. Users close long answers; the custom gateway closes their streams while backend work keeps running. Your job: Fix only handle_disconnect() in gateway.py. Propagate cancellation once for queued and running requests, preserve terminal states, and leave resource retirement to the engine. Investigate: Run python cancellation_monitor.py. The abandoned request generates six unwanted tokens and keeps its allocation, while a second user still wants their answer. The supplied fake engine demonstrates delayed acknowledgment and completion races without a GPU or model download. Proof: Cancellation stops future scheduling; engine-owned acknowledgment releases capacity exactly once. Duplicate disconnects and inline callbacks must be safe, and unrelated users must finish. Run python -m pytest -v, then open a PR against main. Book: Closing the socket stops generation. Prod: Downstream work needs an explicit lifecycle signal, and cancellation requested is not cancellation completed. This exercise repairs a custom adapter using deterministic CPU events. It does not claim real-GPU performance or that a particular serving engine inherently leaks disconnected requests.

P1 — STREAMS CUT DURING ROLLOUT Support: The replacement passed health checks, but 38 of 80 in-flight answers stopped mid-sentence when old workers received SIGTERM. HTTP success stayed green because responses had already started. Your job: Fix only advance_shutdown() in lifecycle.py. Stop new admissions while preserving active generations, then apply an explicit timeout policy at a fixed deadline. Repeated shutdown signals must be safe. Investigate: Run python drain_monitor.py to compare a normal rollout with a stuck generation. The same deterministic CPU replay and fake clock drive the tests—no GPU, model download, or cloud account required. Proof: Normal drain completes both answers; a stuck generation is interrupted only at the deadline; late traffic is refused by the old worker. Run python -m pytest -v, then open a PR against main. Book: A healthy replacement means the old worker can stop. Prod: New-request routing and in-flight stream completion have separate lifecycles. This exercise repairs a custom lifecycle adapter. It teaches readiness, admission, graceful shutdown, and engine-owned retirement using an explicit simulation contract, not real-GPU performance certification.

CHECKPOINT_ROLLBACK_AFTER_REBOOT — visible is not durable Collector Platform: The service logged checkpoint saved: offset=42, and an immediate read returned 42. After a host failure it recovered offset 41 and replayed already-processed events. The trap: Writing a temporary file and renaming it provides atomic visibility, but buffered data and the renamed directory entry may not yet be crash-durable. A successful read-after-write does not prove power-loss persistence. Your task Fix save_checkpoint() in src/lib.rs. Write and flush the temporary checkpoint, synchronize the file, rename it on the same filesystem, synchronize the parent directory, and propagate every error before acknowledging success. CI uses a deterministic Linux-like crash model plus a real temporary-directory test. Book: Rename is atomic, so the checkpoint is safe. Prod: Atomic visibility and durability are separate guarantees.