--- title: "What is context engineering?" canonical: "https://xerj.org/science/context-engineering" updated: "2026-11-14" source: "content/science/context-engineering.md" --- # What is context engineering? **Short answer** (updated 2026-11-14): What is context engineering? It is choosing what an agent actually reads, so the context window is spent on the task. A common loop is to grep and open matching files. A tighter loop retrieves a cited passage and stops. On 8 tasks, the same model used 9,982 output tokens with XERJ against 26,477 when it grepped the source. **Key number:** On 8 tasks, the same Claude Code model used 9,982 output tokens with XERJ against 26,477 when it grepped the source, 2.7 times fewer. (as of 2026-08-06; source: [/case-studies/reference-coding](/case-studies/reference-coding)) **TL;DR** Context engineering is choosing what an agent actually reads. The goal is a short, cited passage in the window, instead of a whole file, so the context window is spent on the task. ## Agent prompt ```text Act as a coding agent. Read https://xerj.org/llms.txt, install XERJ, start a local node, index one folder I point you to, and run one search so I can see a cited passage. ``` *Figure: Byte figures: benchmarks captured on psf/requests, published in the grep-token answer (2026-10-19). Token figures: the reference-coding case study. Do not scale one study into the other.* ## How it works An agent has a limited context window. Every line it reads occupies that window. A common loop is to grep, open every matching file, and hope the useful passage is still in view. A tighter loop retrieves a ranked passage with a file and line, and stops. The agent reads less, and the useful line is not buried in the middle of a long prompt. Liu and colleagues (2023) showed that models often use the start and the end of a long context more reliably than the middle. The paper is [Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172). That result is one reason a shorter, cited passage is worth the extra retrieval step. It is not a XERJ measurement. ## How XERJ uses it XERJ is the retrieval step that makes the tighter loop possible. On [psf/requests, one comparison](/answers/local-code-search-mcp-fewer-tokens-than-grep) recorded 1,303 bytes for a xerj search answer against a 25,150 byte ripgrep listing, measured once, on one repo. Opening the 17 matching files would have been 447,383 bytes. A byte is not a token. On 8 coding tasks, the same Claude Code model used 9,982 output tokens with XERJ against 26,477 when it grepped the source, 2.7 times fewer, and solved 16 of 16 both ways. That study is the [reference-coding case study](/case-studies/reference-coding). Do not scale the byte study into the token study. The product pattern is [reference coding](/science/reference-coding). ## History and research The long-context result cited above is Liu et al., [Lost in the Middle](https://arxiv.org/abs/2307.03172), 2023, later in TACL. The token and byte figures on this page are XERJ's published measurements, from the sources named above. ## Guides Guides: [code search with fewer tokens than grep](/answers/local-code-search-mcp-fewer-tokens-than-grep), [index a monorepo](/answers/index-monorepo-for-agent) and [Claude Code memory](/answers/claude-code-memory). ## Related terms [Reference coding](/science/reference-coding) and [syntax-aware search](/science/syntax-aware-search). ## FAQ ### What is context engineering? It is the work of choosing what goes into an agent's context window: a short, cited passage rather than a whole tree of files. ### Why does this matter? A long context can hide the useful line. Published research on long contexts found models use the start and end of a window more reliably than the middle. ### What has XERJ measured? On 8 tasks, 9,982 vs 26,477 output tokens, from the reference-coding case study. On psf/requests, 1,303 vs 25,150 bytes, once, one repo. A byte is not a token. ### How do I do this with XERJ? Index the folder, search, and read the cited passage. The agent-facing setup is reference coding. The byte comparison is the grep-token answer. ## Evidence - On 8 tasks, the same Claude Code model used 9,982 output tokens with XERJ against 26,477 when it grepped the source, 2.7 times fewer. — [/case-studies/reference-coding](/case-studies/reference-coding) - On psf/requests, one xerj search answer was 1,303 bytes against a 25,150 byte ripgrep listing, measured once. A byte is not a token. — [/answers/local-code-search-mcp-fewer-tokens-than-grep](/answers/local-code-search-mcp-fewer-tokens-than-grep) ## Related - [Reference coding for AI coding agents](/science/reference-coding) - [Reference-coding for Claude Code](/case-studies/reference-coding) - [Local code search MCP that uses fewer tokens than grep](/answers/local-code-search-mcp-fewer-tokens-than-grep) - [What is syntax-aware search?](/science/syntax-aware-search) - [What is the Model Context Protocol?](/science/model-context-protocol) (same intent: Science) - [What is agentic memory?](/science/agentic-memory) (same intent: Science) - [BM25 vs vector search](/science/bm25) (same intent: Science)