Install XERJ (docs: https://xerj.org/llms.txt), index this project's sources, and set up reference coding: clone and index the open-source repos closest to what we're building, and search how they solved a problem before writing code.
One paste — your agent installs XERJ, indexes your project and the open-source repos worth learning from, and looks implementations up instead of re-deriving them.
A coding agent hitting an API it hasn't memorised invents method names,
fails to compile, and loops — spending output tokens, the
expensive kind, on every lap. Grep doesn't save it: grep tells the agent
where to look, and the recovery is still reading source into context —
up to 1.06M input tokens on one corpus in our
measurements. Reference coding flips the loop: clone the open-source
repos closest to what you're building,
xerj autoindex them once, and the agent
retrieves the exact definition — with its contract — before writing.
Measured (8 tasks across 4 languages, 16 runs per arm, real
claude -p token counts): 2.7× fewer
output tokens than grep-driven Claude Code at the same 16/16
solve rate, 26× fewer than working from memory
(260,916 → 9,982), 2.1× cheaper ($1.58 vs $3.27). In a
companion run on a Rust library the model had never seen:
9/9 with retrieval vs 0/9 from memory. The three helper
scripts below ship with the case study — the hero prompt sets them up.
$ xc-corpus.sh xerj-storage https://github.com/spacejam/sled … # clone the peers, once $ xc-index.sh xerj-storage # xerj autoindex, once per corpus $ xc.py xerj-storage "how does sled fsync its WAL segment on rotation?" → the exact function, file:line, contract included — a passage to read, not a tree to grep
THE FULL CASE STUDY — EVERY NUMBER, EVERY TASK, THE EXACT PROMPT →·WHERE THE USERS-REPORT ~5× FIGURE COMES FROM →
The fastest way to make data useful to an AI agent is to not write a
pipeline at all. xerj autoindex <folder>
— a subcommand of the same binary — walks the tree, sniffs every file's
format by content (extensions are never trusted) across
13 format families — JSONL, JSON, dialect-sniffed CSV, logs, SQL dumps,
SQLite, PDF, DOCX, HTML, XML, YAML, plain text, gzip — infers field
types and date encodings from the data itself, writes explicit mappings,
and streams everything in with idempotent IDs. Junk files are recorded,
never fatal. It ends by writing a catalog index — the data map — so the
agent's first question, "what is even in here?", is answered by
the engine. Real captured run:
$ xerj autoindex ./sample autoindex: 4 files (0 MB) under /tmp/xerj-pubpass/sample phase A: sniffing + sampling 4 files… phase A: 3 datasets inferred, 1 junk/skipped files phase B: indexing 3 files with 8 workers → http://localhost:9280 done in 0.2s — 3 datasets, 5801 records live, 0 junk records, 1 junk/skipped files ax-logs 5000 docs ax-exports 800 docs ax-docs 1 docs next: `xerj autoindex map --url http://localhost:9280` for the data map; search via GET /ax-*/_search
Honesty first: in a controlled 10-question exam on that 518 MB corpus, a XERJ-backed agent scored 9 correct + 1 partial vs a fair grep/python baseline's 10/10 — a tie on accuracy, not a rout. What XERJ wins is structural: a full corpus inventory in 4 API calls, sub-second aggregations over millions of rows, and uniform access to SQLite, DOCX, gzip, and decimal-comma CSV through one API — the advantages that matter at scale, over remote/API-only access, and under repeated querying. The pipeline is streaming and resumable, verified on multi-GB corpora.
Every operation an agent needs is a plain HTTP call on :9200 — no SDK, no signup, no external embedding key. Store what an agent learns, recall it later by meaning (the built-in embedder is lexical hash-based — honest hybrid lexical+vector retrieval, not neural understanding). Each namespace is physically isolated, so agents never read each other's memories. Real run against an empty XERJ:
$ curl -sXPOST localhost:9200/_memory/agent-demo \
-H 'content-type: application/json' \
-d '{"text":"The user prefers metric units and a dark UI theme.","metadata":{"kind":"preference"}}'
{"created":true,"id":"77eff57b-e432-431c-8b49-8a16b33ab551","namespace":"agent-demo"}
$ curl -sXPOST localhost:9200/_memory/agent-demo/_recall \
-H 'content-type: application/json' \
-d '{"query":"what display settings does the user like?","semantic":true,"k":1}'
{"hits":[{"id":"77eff57b-e432-431c-8b49-8a16b33ab551",
"score":0.655571460723877,
"text":"The user prefers metric units and a dark UI theme."}], // metadata trimmed
"namespace":"agent-demo"}
xerj autoindex <folder> — sniffed formats, inferred mappings, a self-describing data map, resumable idempotent ingest.
The /_memory REST API — store, recall by meaning / keyword / vector, filter, forget, per-agent isolation.
First-class documentation, each validated end-to-end against a live XERJ — RAG, hybrid search, log analytics, anomaly detection, ES migration.
15 release candidates in 34 days — v1.0.0-rc.1 shipped
2026-07-07, v1.0.0-rc.15 on 2026-08-10, and the installers
above always resolve the latest. Every commit re-runs the
1,365 / 1,368 ES-YAML conformance suite; the changelog
states known defects instead of implying their absence. In the current
build: code arrives AST-parsed for 13 languages (symbols,
definitions, line numbers), index lifecycle policies execute
(ILM / ISM), .gitignore is honoured on
autoindex, and dense-vector kNN is HNSW-served with exact rescoring.
Linux / macOS / Windows × x86-64 / arm64, SHA-256 checksums, prerelease-transparent.
CHANGELOG What changed, honestlyKeep-a-Changelog format — including the defects we know about at cut time.
ROADMAP Where it is goingVerified against the release binary, with public milestones and a live project board.
Every feature exists to cut the thing agents actually spend: context. Reading a file costs a context window; asking an index costs kilobytes. XERJ puts everything an agent needs to ask — code, documents, vectors, memory — behind one query surface.
/_memory REST API — store, recall by meaning / keyword / vector, filter, forget, physically isolated per agent — and a /_graph knowledge layer with evidence on every link.gitignore, records junk instead of crashing on it, and resumes incrementallyCEO for Xerj / [Your Name]