When a backup request backfires

In a two-slot test, a backup after 1.177 ms cut 6 ms deadline misses from 10.1% to 1.3% at 400 requests/s. At 1,600 requests/s, misses rose from 26.9% to 98.8%.

Run the analysis in Colab ยท Read in Markdown

Percent of requests missing a 6 ms deadline at three offered loads. At 400/s, primary only 10.1%, hedged 1.3%. At 1,600/s, 26.9% and 98.8%. At 2,600/s, 98.2% and 99.4%.

What the queue saw

Inspect eight measured requests in Perfetto. Each pair starts at its own planned arrival. Expand the primary and backup lanes to see how long each copy waited for a worker slot. Selection rule and export code.

Two replicas shared two worker slots on one host. Both copies ran to completion. The experiment measured 36,800 requests over 12 runs at three offered loads, with two workload seeds and two policies. The clock started at the planned arrival and stopped when the first copy finished.

Misses at 6 ms, pooled across two seeds
Offered loadRequests per armPrimary missesHedged missesBackups launched
400/s1,600161 (10.1%)21 (1.3%)261
1,600/s6,4001,722 (26.9%)6,324 (98.8%)6,350
2,600/s10,40010,212 (98.2%)10,338 (99.4%)10,371

Recompute the deadline misses in Hugging Face SQL or the DuckDB shell.

At 1,600 requests/s, the timer launched 6,350 backups for 6,400 requests. Those copies competed for the same slots as the primaries. Services with independent capacity or effective cancellation can behave differently.