EVERY NUMBER HERE
TRACES TO A RUN.

FAST WHERE IT'S USED,
QUIET WHERE IT ISN'T.

Two performance stories matter for an engine that lives next to your agents: how fast a query comes back, and what the process costs when nobody is asking. Both are measured; neither is extrapolated.

CORRECT WHERE
MEMORY FAILS.

The retrieval claim is not "search is nice" — it is that retrieval makes the same model correct on APIs it has never memorised. Measured across 13 purpose-built libraries in 5 languages (unfamiliar by construction, so the model cannot bluff), with hidden-test verdicts and real token accounting from claude -p.

• AST-aware indexing, built in. xerj autoindex parses source through tree-sitter grammars in 13 languages and emits every symbol with its kind and line, a searchable defs field, and the full body — so an identifier query ranks the definition first. No ctags, no sidecar toolchain: one binary. Extraction throughput is measured and published (~1,500 files/s single-thread on a 6,113-file Lucene checkout), and an unchanged re-index skips the parse for byte-identical files (~100× on the edit-and-rerun path, shipped in rc.71).
• A passage, not a context dump. Grep-driven agents recover by pulling files into the model's context — up to 1.06M input tokens on one corpus in the measured study. A retrieval answer is the definition with its file:line: a few hundred tokens carrying the one thing a compiler can never leak, the runtime contract.
• Where retrieval is worth nothing — said out loud. On popular public code the model has memorised, bare is cheapest and retrieval is overhead. The case study measures that boundary instead of hiding it; the value is gated by memorisation, which is why it is largest on your private code.

Method, per-library table, and every per-run record: the reference-coding case study · docs/case-studies/reference-coding

FUSE FIRST.
JUDGE WHAT'S LEFT.

Two questions, both answered on public datasets anyone can download. How much does hybrid retrieval add over BM25? And how many routing and spam decisions can labelled history settle with no model call at all? Every vector and hybrid figure in this section was measured with --embed-mode neural and the built-in all-MiniLM-L6-v2 model — not with the default embedder, which is lexical feature hashing and has no model in it. The BM25 rows use no embedder and hold on any node. The Jev rows below are written up verdict-first — wins, losses, and the judge's bill — in the blog post: does XERJ beat JEV?

• The three hosted rows are not a controlled comparison, and carry no verdict. XERJ did not run those systems. They rerank that project's own first-stage shortlist, which is not XERJ's. The dataset and the metric match; nothing else is controlled. What the table supports is narrow: on NFCorpus the local hybrid figure sits inside the range of the published reranker figures, and on SciFact it is lower than all three.
• Fusing with a model scored above reranking with it. Reordering a BM25 shortlist can only move documents BM25 already returned; fusion also admits the ones only the vector list found. On NFCorpus BM25 returned zero hits for 25 of 323 queries, and no reorder repairs an empty list. That is why XERJ ships no local "MiniLM rerank" provider — it would score below the hybrid search that already exists.
• What neural mode costs. In the same run: about 3–7 documents per second to index on CPU (SciFact took about 25 minutes), and about 220–370 ms p50 per hybrid query against about 18 ms for BM25. The machine was also compiling, so read the latencies as upper bounds.
• The rerank stage is now measured — and its first measurement was a disaster we published. With an evaluation key granted by TypeSafe, the stage as it shipped scored 0.3822 nDCG@10 against BM25's 0.7750 on a 40-query pilot — our request asked the judge an untargeted question, so it judged the pile once and echoed query-level noise. Root-caused, fixed (0.8299, raw API 0.8389), then run in full on three datasets: 0.7410 / 0.3312 / 0.3638 over 1,271 judged queries, three repeats each. FiQA is the judge's biggest lift (+0.126 over BM25) and its worst calibration case. The probabilities order documents well but are not calibrated to BEIR relevance. ECE runs 0.10–0.31 across the three datasets, and on FiQA documents rated 0.93 were relevant 34% of the time. Do not min_score the raw number. The full autopsy, run and gate. The stage is also the one XERJ feature that sends document text off the machine — read that before turning it on.

• What it means for you. Where you already have labelled examples, index them and classify new text by a rank-weighted vote over its ten nearest neighbours. On Banking77 the neural arm settled 86.9% of traffic at confidence 0.8 or higher and was right on 0.979 of it, with no tokens spent; only the rest needs a judge model or a person. Spam detection reached 0.983 with plain BM25, under a millisecond per message. AG News shows the scale end: 92% of 7,600 four-way topic decisions right against a 120,000-deep history, 34.2 ms per item. The vote now also speaks TypeSafe's System One wire — POST /v1/systemone on the native port — so the pip-installed jev-reranker runs against a local XERJ node unmodified, zero egress: the write-up.
• The engine under it got 2.0× faster with bit-identical results. A full news item is a ~40-clause disjunction whose match set is most of the index, and the generic search path paid full materialisation for it. A WAND-style pruned walk (Broder et al., the same idea tantivy uses) now scores a document only when it can still make the page. 68.7 to 34.2 ms per item over all 7,600 AG News items, same labels, same confidence bands; the three BEIR BM25 figures reproduce to four decimals and the ES-YAML conformance suite stayed at 1371 passed / 0 failed. Commit cf6a9989.
• What it is not. It is not zero-shot: a new category has no history to vote from, and that case needs a judge model. XERJ did not run any judge model on these datasets and claims nothing about how one would score. Three datasets, one run each.

Scripts, raw output and reproduce commands: benchmarks/beir-hybrid · benchmarks/decisions-as-retrieval · run 2026-09-18 on xerj v1.0.0-rc.74 · AG News 2026-09-20 on the wire-compat branch engine (cf6a9989) · written up in hybrid search quality and decisions without a model call. Hosted-reranker figures: hev/jev-rerank (MIT) · the measured Jev rows and the System One wire: blog/does-xerj-beat-jev.

THE TOKENS YOU
DON'T SPEND.

Output tokens are the expensive kind — priced roughly 5× above input on Claude models — and retry loops on unknown APIs burn exactly those. Three arms of the same agent on the same tasks: memory only, grep-driven, retrieval-injected. Every figure from claude -p --output-format json.

ZERO-TOKEN
ARCHITECTURE.

The cheapest token is the one your model never generates. ZTA is the design rule that produced every number above: spend compute once, at index time, so agents stop spending inference tokens — the metered, per-request, 5×-priced resource — re-deriving what the index already knows. It is an architecture target with measured proxies, not a certification; here are its four principles and the number that keeps each one honest.

Why this is the sales pitch and not a slogan: infrastructure used to compete on latency; agent infrastructure competes on your inference bill. A retrieval that answers in 10 ms and a few hundred tokens replaces a 15,000-token retry loop every time it fires — the $21.90 → $3.38 delta above is that substitution, measured 21 times over. ZTA is the commitment that every future XERJ feature is judged by the same question: how many tokens does it stop your model from spending?

RUN THE NUMBERS ON YOUR CODE.