Prefix Atlas
Source code ↗

Size a prefix cache.

Explore how cache capacity changes page hits on recorded LLM requests. Pick a trace or open a capacity curve from your own workload.

Page hit curve

Atomic prefix lookup
Page hit rate by cache capacityThe horizontal axis shows cache capacity on a logarithmic scale. The vertical axis shows percent of page accesses served from cache.

Cache capacity in pages · log scale

Page hit rate

—

Page hits
—
Page misses
—
Cold pages
—
pages

%

Compare cache budgets

What counts as a hit?

On arrival, each request reuses its longest contiguous cached prefix. After completion, pages are touched from tip to root, leaving shallow pages most recent. The replay starts empty and processes requests in order. It models complete pages and excludes GPU scheduling, decode allocations and concurrent pinning.

Trace details