What's the actual difference between a vector database and full-text search when I have a folder?

TL;DR — Full-text search matches tokens and ranks with BM25, while a kNN query compares one embedding against stored dense vectors. XERJ holds both in one index, so the decision is which clause to send. On our 8 judged queries the three modes disagreed on the top hit once.

The mechanical difference

Full-text search finds documents that contain your tokens and ranks them with BM25. A kNN query ignores tokens, embeds the query into a dense vector, and returns the nearest stored vectors by cosine similarity.

Hybrid search sends both sub-queries in one request and fuses the two ranked lists with Reciprocal Rank Fusion.

XERJ stores the inverted index and the dense_vector field in the same index, so all three are clauses against one endpoint. The default embedder behind the vector clause is lexical feature hashing, and --embed-mode neural is the opt-in that replaces it.

Three queries where the modes differ

Our capture ran 8 judged queries against 20 labeled documents on a single-node host. The judged sets come from the fixture's own published judgments, so this page reports declared misses rather than misses chosen after the fact.

The first row is a recall failure and a precision failure at once. BM25 found the one document containing the literal token car. The vector clause added Coffee bean storage and Inverted index basics instead of the two synonym documents, because the lexical default embedder cannot connect a synonym.

The second row is the one query in 8 where the top hit changes. BM25 put Canine behaviour notes first because that document repeats common words from the question. Both vector-backed modes put Checkpoint journal replay first, one of the two judged documents.

The third row shows the padding effect. BM25 stopped at 2 hits because only 2 documents contain the tokens. The vector clause always returns k results, so it filled position 3 with Capacity planning.

What the aggregate says

Precision at 3 and recall over all 8 queries, on the same index and the same judgments.

No mode wins everywhere. Weighted linear fusion was best here, and it was still wrong on more than half the judged documents at k of 3.

Approximate kNN is opt-in, not automatic

XERJ ships a real HNSW graph, and it uses that graph only when seven conditions hold at once. The seven conditions are these.

1. Cosine similarity only.
2. No filter on the kNN clause.
3. Not a nested field.
4. No passage-chunk exactness requirement.
5. A graph pinned to the queried field.
6. No scalar8 quantization.
7. At least 1,024 vectors, with full coverage.

Anything else falls back to exact brute force, silently. A filtered kNN query, an l2_norm similarity, a nested field or an index of 900 vectors all take the exact path.

Two details soften that. Even on the approximate path XERJ exact-rescores every candidate in f64, so _score matches brute force. The ef parameter is floored at 800, because a literal ef of 100 measured recall at 10 of 0.53 against 0.937.

Our capture built two indices for this boundary: route_big with 1,536 vectors and route_small with 512. No field in any kNN response names the route it took, and profile: true adds none, so a reader cannot confirm the path from the response.

The kNN request needs a literal vector

XERJ rejects the Elasticsearch query_vector_builder idiom with an HTTP 400 and needs a literal embedding in query_vector. No query-time text-embedding route exists.

The supported route is two requests. Index the query text into a semantic_text field, then read the companion <field>_vector value out of _source and paste it into the kNN clause.

parse error: `knn` requires `query_vector` (or `vector`) as a float array

How to choose

Choose full-text search when your users type words that appear in the documents. BM25 also gives you exact counts and an inspectable term match.

Choose the vector clause when queries and documents share meaning but not tokens. Start the node with --embed-mode neural before you rely on that.

Choose hybrid when both are partly right. XERJ is single-node, with no replication and no managed service. A deployment that needs multi-node vector serving needs a different product.

What this capture does not show

The corpus is 20 documents and the judged set is 8 queries, run once each on one shared host. The content map asked for 3 queries with 3 different top hits and the run produced 1. This page publishes the returned sets rather than a stronger claim.

FAQ

What's the actual difference between a vector database and full-text search?

Full-text search matches tokens and ranks with BM25. Vector search compares one embedding against stored dense vectors. XERJ serves both from one index.

Do I need a vector database for this, or am I overcomplicating it?

For one folder on one host it is usually more parts than the job needs. XERJ keeps the dense vector beside the inverted index, and one _search request can query either or fuse both.

Do I even need embeddings for search, or is full-text enough?

Full-text is enough when people type words that appear in the files. On our 8 judged queries BM25 scored 0.375 precision at 3 and kNN 0.4167, both on the lexical default embedder.

When should I combine keyword search and vector search?

When both are partly right. Weighted linear fusion scored 0.4583 precision at 3 on the same judged set, the best of the 4 modes, and it was still wrong on more than half the judged documents at k of 3.

When does BM25 beat vectors?

When the query token is present in the document. For quokka BM25 returned exactly the one judged document and nothing else.

When does kNN beat BM25?

When no token matches. On our restart question BM25 ranked an irrelevant document first and kNN ranked a judged-relevant document first.

Does XERJ use approximate kNN?

Only when seven conditions hold, including cosine similarity, no filter and at least 1,024 vectors. Everything else runs exact brute force.

Related

• How do I build RAG without a vector database?
• When should I turn on neural embeddings, and when is the lexical default enough?
• Should an agent use API search or vector search?
• Should my agent call a search API or stand up Qdrant for persistent memory?