HOME·ANSWERS·WHAT IS LOCAL RAG?
ANSWERS · ANSWERS FOR CODING AGENTS

What is local RAG?

AUTHOR · XERJ documentation team · REVIEWED BY · XERJ engineering team · PUBLISHED · 2026-11-13 · UPDATED · 2026-11-14
SHORT ANSWER · UPDATED 2026-11-14

What is local RAG? It is retrieval-augmented generation that keeps the corpus on your machine. You index the files, retrieve cited passages, then a model answers from those hits. XERJ does the retrieval step in one Apache-2.0 binary, with no separate vector database and no embedding API key.

Key number: On main, xerj mcp registers 24 MCP tools, or 21 when the binary has no Corpus Hub key. (as of 2026-10-28 · source: landing/docs/agents/schemas/mcp-tools.json)

TL;DR Local RAG means the corpus stays on your machine. You index the files, retrieve cited passages, then a model writes an answer from those hits. With XERJ the retrieval step is one Apache-2.0 binary: xerj autoindex, then xerj search or a hybrid query. There is no separate vector database. The default embedding mode is lexical, not neural.

Try it: a local RAG loop

Index locally, retrieve, then generate with citations

  1. 1. Install and start a node

    curl -fsSL https://xerj.org/get | sh, then xerj --insecure --data-dir ./.xerj-data. Put the data dir outside the folder you will index.

  2. 2. Index the corpus

    xerj autoindex ./docs --url http://127.0.0.1:9200 --prefix docs --progress plain. No schema and no separate vector database.

  3. 3. Retrieve with citations

    xerj search, a hybrid query, or POST /docs-*/_search. Each hit carries ax_path or file:line. The default embedding mode is lexical, not neural.

  4. 4. Generate from the hits

    Hand the passages to your model and require it to cite those paths. If a fact is not in a hit, say so. Nothing in this loop leaves the machine unless you send it.

This is the local loop on one binary. It is not a new measured RAG cost study. Query-1 compared with RAG lives on its own page.
PASTE THIS TO YOUR AI AGENT — IT DOES THE REST Act as a coding agent. Read https://xerj.org/llms.txt, start a node with `xerj --insecure --data-dir ./.xerj-data`, index the folder you were handed with `xerj autoindex ./docs --url http://127.0.0.1:9200 --prefix docs --progress plain`, retrieve with `xerj search`, and write the answer only from cited hits. If a fact is not in a hit, say so.
RUN THIS XERJ COMMAND curl -fsSL https://xerj.org/get | sh Install the release binary into ~/.local/bin.
RUN THIS XERJ COMMAND xerj --insecure --data-dir ./.xerj-data Start a local development node. Put the data dir outside the folder you will index.
RUN THIS XERJ COMMAND xerj autoindex ./docs --url http://127.0.0.1:9200 --prefix docs --progress plain Build a local index. No schema and no separate vector database.
RUN THIS XERJ COMMAND xerj search "what is the default timeout" Retrieve cited passages. Hand only those hits to the model.

The loop

Retrieval-augmented generation has two parts. Retrieval finds passages. Generation writes an answer from them. Local RAG keeps both the files and the index on your machine. The model should cite the paths it was given, and should say so when a fact is not in those hits.

XERJ is the retrieval step: one Apache-2.0 binary, no separate vector database, no embedding API key. The same design is explained in RAG without a vector database. A local AI search engine, without the generation step, is What is a local AI search engine?.

Why this saves context window space is What is context engineering?. Published byte and token studies of retrieval against printing matches are on the retrieval studies page.

How to run it

  1. Install the binary: curl -fsSL https://xerj.org/get | sh.
  2. Start a node: xerj --insecure --data-dir ./.xerj-data. Put the data dir outside the folder you will index.
  3. Index the corpus: xerj autoindex ./docs --url http://127.0.0.1:9200 --prefix docs --progress plain.
  4. Retrieve: xerj search "what is the default timeout", a hybrid request over the REST API, or POST /docs-*/_search.
  5. Generate only from those hits, and keep the ax_path or file:line next to each claim.

An agent can do the same retrieval through the MCP server. The protocol is What is the Model Context Protocol?. Ranking options are What is hybrid search? and What is semantic search?.

The default embedding mode is lexical, not neural. Start the node with --embed-mode neural when you want the built-in neural embedder.

What this page does not measure

This page is a how-to. It publishes no new RAG cost or accuracy study. The Query-1 model, which compiles a prompt to a verifiable search program, is compared with RAG on the Query-1 page. Read that page for those figures. Do not copy them into a new breakdown here.

Read next

Cohere RAG with XERJ, reranking, grounded answers and a local AI search engine.

FAQ

What is local RAG?

It is retrieval-augmented generation that keeps the corpus on your machine. You retrieve cited passages from a local index, then a model writes an answer from those hits.

Do I need a separate vector database?

Not with XERJ. The embedder ships in the binary. See RAG without a vector database for the same point, with more API detail.

Is there a local RAG MCP server?

xerj mcp is the MCP server in the same binary. On main it registers 24 tools, or 21 with no Corpus Hub key. The agent retrieves from the local index, then writes from the cited hits.

What is the default embedding mode?

Lexical, not neural. Semantic and hybrid queries still run, but they use word and sub-word overlap plus BM25 unless you start the node with --embed-mode neural.

How is this different from Query-1?

Local RAG still uses a language model to write the final answer from retrieved hits. The Query-1 model compiles a question into a program the engine runs over your facts. That comparison lives on the Query-1 page.

Evidence

Related