Where do your prompts split?
Two agents load the same tools in a different order. The second request can lose a reusable prefix. CachePin finds the first change and shows what a consistent order would recover.
Compare two prompts ↓ Run the CLI example ↗Prefix comparison
Choose a pair or edit the JSON. This browser example compares the serialized model, tools and system fields in UTF-8 before and after normalization. It omits messages. Use the CLI for a full request diagnosis and token estimates.
Compare requests in your MCP client
Open the CachePin bundle in a client that supports MCPB extensions on macOS or Linux (x86-64 or ARM64). Call diagnose_diff with two serialized requests, prev_json and curr_json. The tool runs locally and returns the byte offset where the requests first differ.
Estimate a cache bill
Change the workload and compare two scenarios. The model assumes the normalized workload pays for one prefix write. Actual hits depend on your provider, timing and cache capacity.
Assumes 512 fresh uncached tokens/turn per agent. Hit-rate penalties for tool order, timestamps and document order are fixed assumptions in this simulator.
Recorded measurements
8-GPU vLLM Eviction-Regime Routing [SIM-TRACE]
48 shared prefixes across 8 GPUs (20,480 KV tokens/GPU, Zipf-1.25 multi-agent bursts, 160 ms decode residency). Single-home 2×-divert regresses P99 TTFT by +9.8% vs Least-Loaded; cachepin cuts P99 by 37.0% and Mean TTFT by 2.42× at ~1.15 µs/route.
| Routing Policy | Fleet KV Hit | P50 TTFT | P99 TTFT | Mean TTFT | Replications | Status |
|---|---|---|---|---|---|---|
| Round-Robin | 51.1% | 136.6 ms | 443.0 ms | 137.3 ms | 366 | KV thrashing |
| Least-Loaded | 52.6% | 106.0 ms | 376.0 ms | 119.1 ms | 355 | Cache-blind |
| Consistent-Hash | 96.1% | 16.8 ms | 351.6 ms | 72.9 ms | 0 | Hot-GPU queue |
| SingleHome-2xDivert (Ranvier-style) | 66.9% | 14.4 ms | 413.0 ms | 89.7 ms | 180 | +9.8% P99 cliff |
| cachepin Adaptive-Replication | 87.5% | 12.8 ms | 260.0 ms | 37.1 ms | 66 | -37.0% P99 · 1.15 µs |
Reproducible Cross-Language Controls [LIVE-CPU]
Measured on identical 18,535-byte multi-agent payloads (benchmarks/compare_controls.py → benchmarks/results.json). Live-hydrated directly from results.json.
| Runtime | Compile P50 | Compile P99 | LCP Diff GB/s | CLI Cold Start |
|---|---|---|---|---|
| cachepin (Rust + SIMD) | — | — | — | — |
| cachepin Python FFI (libcachepin.so) | — | — | — | — |
| Node.js 22 (V8 JIT control) | — | — | — | — |
| Python 3.12 (stdlib control) | — | — | — | — |