Verify claims in a report against the saved sources
Verify claims in a report against the saved sources. Index those files, then run one match_phrase per claim. A claim with no hit is unverified. On 2026-10-20, 3 of 5 phrases were in the docs index and 2 were not. The first check took 490 ms.
Key number: 3 of 5 body phrases were found in the docs index; the first check took 490 ms. (as of 2026-10-20 · source: benchmarks/agent-queries-2026-10-20/08-claims.jsonl)
TL;DR — Verify claims in a report against the saved sources. Index those files, then run one match_phrase per claim. A claim with no hit is unverified. On 2026-10-20, 3 of 5 phrases were in the docs index and 2 were not. The first check took 490 ms.
Read https://xerj.org/llms.txt. Start one local node, index the saved source files, and run one match_phrase query per claim on the prose field from xerj autoindex map. Mark a claim found only when a hit comes back, and name ax_path. A miss means the words were not in that field.
xerj autoindex --url http://127.0.0.1:9217 --yes --no-graph --progress plain --disable-feedback /workspace/docs --prefix docs --state-dir /tmp/xerj-aq-2026-10-20/state-docs
Index the saved sources. This docs folder exited 3 with 9,940 records.
curl -s -XPOST http://127.0.0.1:9217/docs-docs/_search -H 'content-type: application/json' -d '{"size":1,"_source":["ax_path","title"],"query":{"match_phrase":{"body":"the primary user is an AI agent"}}}'
One claim, one phrase query. Found in WHY_XERJ.md, took 490 ms.
curl -s -XPOST http://127.0.0.1:9217/docs-docs/_search -H 'content-type: application/json' -d '{"size":1,"_source":["ax_path"],"query":{"match_phrase":{"text":"the primary user is an AI agent"}}}'
The same words on the wrong field. 0 hits, took 0.
XERJ is a new way for AI to search for data.
One phrase, one claim
The capture indexed docs/ (wall 14.3 s, exit 3, 9,940 records) and queried docs-docs, whose prose field is body.
curl -s -XPOST http://127.0.0.1:9217/docs-docs/_search -H 'content-type: application/json' -d '{"size":1,"_source":["ax_path","title"],"query":{"match_phrase":{"body":"the primary user is an AI agent"}}}'
Found. ax_path was WHY_XERJ.md. took was 490.
The next phrase, shares no code and no architecture, was found in ARCHITECTURE.md (2 hits) and took 485 ms. point-at-folder was found in WHY_XERJ.md and took 1 ms.
What a miss means
Two phrases were not found, each with took 1.
Content sniffing never trusts the extension is a rewording. The file does not contain that sentence. The check correctly refused it.
XERJ is a fork of Lucene is not in the file either. The check refused it too. A miss does not say the sentence is false in the world. It says these words are not in the field you searched.
The same five phrases against text on the same index returned 0 hits, and each took was 0. Wrong field, silent empty. xerj autoindex map is how you learn the field name.
Measured on 2026-10-20
Phrase on body | Result | took |
|---|---|---|
| the primary user is an AI agent | Found, WHY_XERJ.md | 490 ms |
| shares no code and no architecture | Found, ARCHITECTURE.md, 2 hits | 485 ms |
| point-at-folder | Found, WHY_XERJ.md | 1 ms |
| Content sniffing never trusts the extension | Not found | 1 ms |
| XERJ is a fork of Lucene | Not found | 1 ms |
Same five phrases on text | 0 hits each | took 0 |
The first two checks were slower than the three that followed. Those are the measured times, not a single combined figure. One node, binary v1.0.0-rc.97.
A phrase match does not prove a claim. It shows the words are present.
When not to use XERJ
If the report paraphrases, or writes a number in a different form, this check will miss a true sentence. Use it for claims you can quote, and treat a miss as "not in this field", not as a verdict.
FAQ
Verify claims in a report against sources?
Index the sources you saved. Run one match_phrase per claim on the prose field. A hit names ax_path. No hit means unverified.
Does a hit prove the claim is true?
No. It proves those words occur together in the field. The sentence can still be a bad reading of the file.
What about a reworded claim?
This capture missed Content sniffing never trusts the extension even though the file talks about sniffing. The words have to be the words on disk.
Why did the same phrase miss on text?
On docs-docs the sentence was in body. The same five phrases on text returned 0 hits. Read the map before you pick the field.
Is this a faithfulness score?
No. There is no score for how well a report follows its sources. You get found or not found, plus the path.
Evidence
- The first body phrase took 490 ms and the second took 485 ms. —
benchmarks/agent-queries-2026-10-20/08-claims.jsonl - Three later body phrases took 1 ms each. —
benchmarks/agent-queries-2026-10-20/08-claims.jsonl - The docs folder index finished in 14.3 s. —
benchmarks/agent-queries-2026-10-20/02-autoindex-docs.txt