Size a prefix cache.
Explore how cache capacity changes page hits on recorded LLM requests. Pick a trace or open a capacity curve from your own workload.
Page hit curve
Atomic prefix lookupCache capacity in pages · log scale
Page hit rate
—
- Page hits
- —
- Page misses
- —
- Cold pages
- —
pages
%
Compare cache budgets
What counts as a hit?
On arrival, each request reuses its longest contiguous cached prefix. After completion, pages are touched from tip to root, leaving shallow pages most recent. The replay starts empty and processes requests in order. It models complete pages and excludes GPU scheduling, decode allocations and concurrent pinning.