HOME·SCIENCE·BM25 VS VECTOR SEARCH
SCIENCE · RETRIEVAL

BM25 vs vector search

AUTHOR · XERJ documentation team · REVIEWED BY · XERJ engineering team · PUBLISHED · 2026-11-13 · UPDATED · 2026-11-13
SHORT ANSWER · UPDATED 2026-11-13

What is the difference between BM25 and vector search? BM25 ranks a document by query-term frequency, how rare those terms are, and document length. A vector query ranks by meaning, so a paraphrase can still match. XERJ uses BM25 by default, and can add a vector arm in hybrid search.

TL;DR BM25 vs vector search comes down to shared terms versus meaning. BM25 scores a document by query-term frequency, how rare those terms are, and document length. A vector query can match a paraphrase that uses different words.

Try it: what the score uses

Three ideas inside BM25

  1. 1. Term frequency

    A document that uses the query term more often gets a higher score, with a cap so repetition does not take over.

  2. 2. Inverse document frequency

    A term that appears in few documents is worth more than a term that appears in almost every document.

  3. 3. Length

    A long document is not automatically better. BM25 adjusts for document length so a short, on-topic passage can still rank well.

  4. 4. In XERJ

    BM25 is the default lexical ranker. You can use it on its own, or fuse it with a vector arm in hybrid search.

The three parts are the usual account of BM25 in Robertson and Zaragoza, The Probabilistic Relevance Framework: BM25 and Beyond (2009). This page does not run BM25 on a new corpus.
PASTE THIS TO YOUR AI AGENT — IT DOES THE REST Act as a coding agent. Read https://xerj.org/llms.txt, install XERJ, start a local node, index one folder I point you to, and run one search so I can see a cited passage.

How it works

Three ideas sit inside a BM25 score.

Term frequency: a document that uses a query term more often gets a higher score, with a cap so repetition does not take over.

Inverse document frequency: a term that appears in few documents is worth more than a term that appears in almost every document.

Length: a long document is not automatically better. BM25 adjusts for document length so a short, on-topic passage can still rank well.

The walkthrough above is a teaching account of those three ideas. It does not run BM25 on a new corpus.

How XERJ uses it

BM25 is the default lexical ranker in XERJ. You can use it on its own, or as the lexical arm of hybrid search. Product examples use k1 = 1.2 and b = 0.75.

For a side-by-side with vector search, see BM25 vs vector search. For the how-to on combining both arms, see how to run hybrid search in XERJ.

History and research

A standard reference is Robertson and Zaragoza, The Probabilistic Relevance Framework: BM25 and Beyond, Foundations and Trends in Information Retrieval, 2009. That survey is the source for the three-part account on this page.

This page does not rest on a second historical paper. It also does not treat a XERJ benchmark as a BM25 research result.

Related terms

Hybrid search, semantic search, and hybrid vs vector vs semantic search.

FAQ

What does BM25 stand for?

Best Match 25. It is a family of ranking functions from the Okapi retrieval work, later set out in detail by Robertson and Zaragoza.

What three ideas does the score use?

Term frequency, with a cap so repetition does not take over. Inverse document frequency, so rare terms count more. A length adjustment, so a long document is not automatically better.

How does XERJ use BM25?

BM25 is the default lexical ranker. You can use it alone, or fuse it with a vector arm in hybrid search. The default k1 and b values used in product examples are k1 = 1.2 and b = 0.75.

How does BM25 differ from vector search?

BM25 needs shared terms and is strong on exact names. A vector query can match a paraphrase. Hybrid search runs both and fuses the lists.

Related