Answers

Direct answers about XERJ search, indexing and agent workflows, with evidence links and practical guidance for AI builders.

Agent access and memory

• Can ChatGPT search a folder on my laptop, or do I need something else? — xerj mcp is a stdio MCP server that proxies 10 tools to a running XERJ node, so an agent reads an indexed local folder instead of opening files itself.
• I need my agent to find a passage and cite the file, not dump the whole tree. How? — Index the repository with xerj autoindex, then call xerj_search through xerj mcp. One captured call returned the file and line for 2 symbols without opening a file.
• Should an agent use API search or vector search? — One question through 4 XERJ MCP tools returned 3 different result sets. Hybrid RRF returned 6 documents and merged both lists on the default lexical embedder.

Agent memory: link retrieval

• Should agent memory use a knowledge graph? — XERJ builds a bounded link index over indexed documents, not a graph database. Our capture found 7 edges, answered 1 and 2 hops, and refused 3.

Agent memory: model dependency

• How do I stop an agent memory store from filling with near-duplicate notes? — Send dedup true on the store call and XERJ returns the existing id instead of a second document. The write path runs no model and made no outbound call.

Agent memory: provenance

• How can agent-memory links retain evidence? — Every detected link in XERJ carries the exact line that taught it. All 7 edges in our capture had an evidence quote, and unlink wrote a tombstone.

Agent memory: security

• How do I give two agents private memory on one laptop? — XERJ keeps agent memory on local disk and made 0 outbound connections in our capture. Roles are stored but not enforced, so scoped API keys do the work.

Agent memory: setup

• How do people store user preferences so an agent can recall them next week? — XERJ keeps agent memory in a reserved index inside the data directory. We stored two memories, restarted the node, recalled both, then forgot one by id.

Agent memory: storage

• How do I set the MCP memory storage path? — XERJ has no separate memory path. Agent memory lives in the node data directory as .xerj-memory-{namespace}, and the MCP server stores nothing itself.

Agent memory: topology

• What's a simple way to give an agent long-term memory on my laptop without Qdrant? — Our capture ran agent memory on one XERJ process with three loopback sockets, no second data service, and a peak resident size of 100,928 kB.

Code security

• How do I audit a PHP codebase with an index instead of grepping 1,400 files? — The published WordPress core audit indexed the tree as queryable facts, then read only the survivors. Core came back hardened, and that negative is the result.

Coding agents: reference retrieval

• How do I set up reference coding so the agent retrieves a mechanism before writing code? — Clone the library you are about to write against, index it with xerj autoindex, and retrieve the passage first. The published study, with the tracks where it loses.

Embeddings: cost

• How do I reduce embedding API cost? — XERJ embeds documents inside the node by default, so a captured lexical run made 0 outbound calls. Proxy mode sent exactly 2 calls for the same 100 documents.

Embeddings: local execution

• I want local search that works offline, no cloud embeddings. What are my options? — XERJ embeds locally in two ways. The default lexical embedder needs no model at all, and --embed-mode neural loads a cached MiniLM model on the CPU.

Embeddings: selection

• When should I turn on neural embeddings, and when is the lexical default enough? — Embeddings only helped when we opted into the neural embedder. On XERJ's lexical default, kNN promoted coffee storage above the synonyms for the query car.

Files and formats

• How do I find every place a config key is set across YAML and XML in a repo? — XERJ autoindex indexes YAML and XML config files as full-text documents, maps 9 fields including body and title, and answers one bool query across both.
• How do I full-text search a SQLite database I just copied onto disk? — xerj autoindex reads a SQLite file read-only and turns each table into an index. A captured WAL-mode run indexed 400 of 450 committed rows and changed no bytes.
• How do I index a CSV with many columns? — xerj autoindex infers a mapping for a wide CSV with no schema work. A captured 120-column file produced 128 mapped fields and an exact terms aggregation.
• How do I index Markdown into the Elasticsearch API? — XERJ autoindex indexes Markdown as the txt-prose family and answers the Elasticsearch API, but it maps no headings array and keeps the hash in the title.
• How do I run syntax-aware code search? — XERJ indexes source files with a defs field and a symbols array, so one query finds declarations with kind, name and line while another finds raw text.
• How do I search a .csv.gz or .json.gz without unzipping it first? — Gzip is a wrapper, not a family. autoindex unwraps it and sniffs the inner CSV or JSON, so a compressed export becomes typed fields rather than one opaque blob.
• How do I search a Confluence export? — Point xerj autoindex at an unzipped Confluence HTML space export. A captured run kept page titles, nested page bodies, and the attachment tree.
• How do I search a Slack JSON export? — Point xerj autoindex at an unzipped Slack export. A captured run found a threaded reply, kept ts, thread_ts and user, and round-tripped Japanese text.
• How do I search BVH motion-capture files? — xerj autoindex detects the bvh family and extracts frames, frame_time_s, duration_s, joint_count and a joints keyword array. The family produces no body field.
• How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them? — XERJ autoindex decompresses a .gz log once at index time. In our capture the plain and gzip copies returned identical counts, identical hits and one date field.
• How do I search Kubernetes pod logs? — Collect the pod logs to disk first, then index the folder. A captured run answered 1 phrase query across 3 pods and returned 18 hits, 6 per pod.
• How do I search through a folder of contracts in .docx? — Index a folder of DOCX contracts with one xerj autoindex command and search their paragraphs. A captured run also shows the decompression guard refusing a file.
• How do I search Unity YAML scenes and the C# that references them as one project? — xerj autoindex detects the unity and unity-meta families, splits a scene into one index per Unity class, and joins an asset to its .meta record by path.
• How do I stop my agent from reading the whole repo into context? — Index the tree once with xerj autoindex, then let the agent query the defs and symbols fields instead of opening files. A dry run first prints every ignore rule.
• How should an agent figure out what's in a messy data folder before searching? — Point xerj autoindex at the folder, then read xerj autoindex map: every dataset it inferred, the fields and types, the time range, and every file it refused.
• I downloaded a bunch of papers as PDFs. How do I search across them? — One xerj autoindex command indexes the folder, then a match_phrase query reads every PDF. A captured run hit all 3 born-digital files and refused a scanned one.
• I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres? — xerj autoindex parses a MySQL-style .sql file directly, with no database server, and makes every INSERT value searchable. A captured run indexed 321 documents.
• I have a directory of CSV exports. How do I query them without opening each one in Excel? — Point xerj autoindex at the folder once. A captured run split 3 CSV files across 2 schemas into 2 indices and answered one query across both of them.
• I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once? — Index a folder of PDFs, Word docs and markdown with one xerj autoindex command, then search the contents. A captured 6-file run indexed 51 documents into 2 indices.
• I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search? — Index the exported ticket JSON on disk. A captured run returned 36 description hits with per-ticket locators and an exact status count of 30 per status.
• I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them? — A captured run indexed one document per notebook cell, with cell_type, execution_count and the nbformat cell id, so a hit names the exact cell.
• I saved a help-center as HTML. How do I search it like the real help-center search? — XERJ autoindex indexes a static HTML export from local disk, extracts title and a headings array per page, and fetched 0 non-loopback peers in our capture.
• Is it efficient to index PDF files? — PDF extraction is the slowest family XERJ parses. XERJ runs it in an isolated worker sized from free memory, caps it 5 ways, and refuses image-only files.
• Is there a way to full-text search my browser history file itself? — Copy the Chrome History SQLite file and index it. A captured run made titles and URLs queryable, left the source unchanged, and missed 40 WAL rows.
• My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way? — Index the spec file itself. A captured run found route names, operation descriptions and operationIds in both the JSON and the YAML serialization.
• What's the best way to search markdown notes and PDFs in the same vault? — A captured XERJ run indexed a vault of markdown notes and returned text from inside a PDF attachment and a DOCX attachment, plus wikilink edges between the notes.
• What's the easiest way to search JSON logs plus some old gzip text logs in the same folder? — xerj autoindex detects JSON arrays, JSONL logs and gzipped logs in one folder, infers the date field and answers exact filters. One run returned 386 ERROR hits.
• Why does my Notion export duplicate titles? — A captured run indexed a Notion Markdown export twice and got the same 8 documents with byte-identical ids. The duplicates come from the export layout.

Folder indexing

• I want to point something at a folder and then ask questions about what's in it. What should I use? — Index the folder with one command, read the map of what XERJ found, then let an agent query it over MCP instead of opening the files one by one.

Hybrid retrieval: architecture

• How do I build RAG without a vector database? — Build a RAG retrieval layer on one XERJ index. One hybrid request fuses BM25 and kNN with Reciprocal Rank Fusion, and no second service runs.

Hybrid retrieval: RRF

• What is Reciprocal Rank Fusion? — Reciprocal Rank Fusion scores a document by 1 divided by k plus its rank in each list. XERJ uses k 60, and this page checks the arithmetic by hand.

Hybrid search

• How does XERJ combine BM25 and kNN? — One hybrid request carries a BM25 sub-query and a kNN sub-query against one index, and Reciprocal Rank Fusion merges their ranked lists into a single result list.

Operations: completeness

• My codebase indexer says indexed but I don't see my code. How do I check it actually finished? — Compare 4 numbers: files on disk, the dry-run plan, autoindex status, and the index count. A captured run cut 64 files to a 3-file plan, naming a rule per gap.

Operations: estimation

• How do I estimate folder-indexing time? — XERJ measures a client-side extraction floor during a dry run and can stop at a decision gate with exit 4. On 2,500 PDFs the floor was 2.2 min, the run 157.7 s.

Operations: exclusions

• Why would a folder search miss files that I can see on disk? — XERJ prints one line per ignore rule and stores a refusal reason per file. A capture reconciles 26 files on disk to 1 indexed, 2 refused and 23 excluded.

Operations: exit codes

• The indexer exited 3 (or 4). Did it fail? — xerj autoindex exits 0 complete, 3 completed-with-junk, 4 needs-a-decision, 2 usage and 1 on an endpoint or journal failure. Only 4 is asking you a question.

Operations: installation

• Is there a single-binary local search engine I can just run? — XERJ runs from one native binary with no container and no JVM. A measured capture records 3 linked libraries and a 1.071 s cold start to the first HTTP 200.

Operations: low-volume logs

• What can I use for low-volume log search? — XERJ indexes plain-text logs on one node with exact error counts and sub-millisecond filters. A measured ladder stops at 16 MB: memory grows about 200x the corpus.

Operations: progress

• How do I read autoindex progress? — XERJ prints xerj-bar, xerj-progress and a final xerj-done line. On a single large file the percentage stays at 0.0%, so read since_progress_s and waiting_on.

Operations: recovery

• The indexer died overnight. Do I have to start over? — XERJ keeps a resume journal in the state directory, so re-running autoindex is idempotent. A capture reached 82,231 documents and matched a clean control run.

Operations: reindex

• Reindexing my files is taking forever. What usually causes that? — A measured XERJ _reindex moved 100,000 documents in 20,836 ms on one node. Read the batches, created and failures fields to tell slow progress from no progress.

Product identity

• What is XERJ? — XERJ is an Apache-2.0 single Rust binary that indexes a local folder with one command and answers search, agent memory and MCP calls from the same process.

RAG: troubleshooting

• How do I improve a basic FAISS RAG pipeline? — Move the chunk store into one XERJ index and fuse BM25 with kNN. Two request shapes that a FAISS user reaches for first both return HTTP 400.

Retrieval behaviour

• Why did my filtered vector search get slower when I added a filter? — A filter takes kNN off the HNSW graph and onto an exact scan whose cost scales with vectors scanned. This page explains when that happens and publishes no timing.

Retrieval: selection

• What's the actual difference between a vector database and full-text search when I have a folder? — Full-text search matches tokens, kNN matches vectors, hybrid fuses both. XERJ serves all three from one index over an indexed folder, so the choice is a clause.