Count mentions of a word over time in a posts file
Count mentions of a word over time in a posts dataset. Index the JSONL, check the date field, and send one date histogram. On 2026-10-20 the phrase autoindex in issue titles returned 88 hits and took 123 ms: 1 in July, 50 in August, 11 in September, 26 in October. A fractional epoch field returned no months.
Key number: A date histogram for the phrase autoindex returned 88 hits and took 123 ms. (as of 2026-10-20 · source: benchmarks/agent-queries-2026-10-20/08-histogram-phrase.json)
TL;DR — Count mentions of a word over time in a posts dataset. Index the JSONL, check the date field, and send one date histogram. On 2026-10-20 the phrase autoindex in issue titles returned 88 hits and took 123 ms: 1 in July, 50 in August, 11 in September, 26 in October. A fractional epoch field returned no months.
Read https://xerj.org/llms.txt. Start one local node, run xerj autoindex on the JSONL folder, and read xerj autoindex map before you aggregate. Send one _search with size 0, a match_phrase, and a date_histogram on the field the map types as date. If that field is double, say the histogram is empty and do not invent months.
xerj autoindex --url http://127.0.0.1:9217 --yes --no-graph --progress plain --disable-feedback /tmp/xerj-aq-2026-10-20/issues --prefix gh --state-dir /tmp/xerj-aq-2026-10-20/state-gh
Index the posts file. This capture used a GitHub issue export, 1,532 records, wall 7.2 s.
curl -s -XPOST http://127.0.0.1:9217/gh-jsonl/_search -H 'content-type: application/json' -d '{"size":0,"query":{"match_phrase":{"title":"autoindex"}},"aggs":{"per_month":{"date_histogram":{"field":"createdAt","calendar_interval":"month"}}}}'
Phrase plus one bucket per month. The first run took 123 ms and returned 88 hits.
curl -s -XPOST http://127.0.0.1:9217/epoch-jsonl/_search -H 'content-type: application/json' -d '{"size":0,"query":{"match_all":{}},"aggs":{"per_month":{"date_histogram":{"field":"ts","calendar_interval":"month"}}}}'
Same aggregation on a double field. 40 docs, zero buckets.
XERJ is a new way for AI to search for data.
Index the file, then read the field type
The measured posts file is a GitHub issue export, not a Reddit archive. createdAt on gh-jsonl is date. The run committed 1,532 records in 7.2 s, exit 0.
xerj autoindex --url http://127.0.0.1:9217 --yes --no-graph --progress plain --disable-feedback /tmp/xerj-aq-2026-10-20/issues --prefix gh --state-dir /tmp/xerj-aq-2026-10-20/state-gh
xerj-done ok=true exit=0 reason=completed wall=7.2s files=2 records=1532 generation=1
A string field becomes date when at least 95% of the sampled values parse as dates. That rule is in infer/mod.rs. Read xerj autoindex map on your own file. Do not assume the column you call created is date.
One query, one count per month
curl -s -XPOST http://127.0.0.1:9217/gh-jsonl/_search -H 'content-type: application/json' -d '{"size":0,"query":{"match_phrase":{"title":"autoindex"}},"aggs":{"per_month":{"date_histogram":{"field":"createdAt","calendar_interval":"month"}}}}'
{"took":123,…,"hits":{"total":{"value":88,"relation":"eq"},…},"aggregations":{"per_month":{"buckets":[{"key_as_string":"2026-07-01T00:00:00.000Z","doc_count":1},{"key_as_string":"2026-08-01T00:00:00.000Z","doc_count":50},{"key_as_string":"2026-09-01T00:00:00.000Z","doc_count":11},{"key_as_string":"2026-10-01T00:00:00.000Z","doc_count":26}]}}}
size 0 asks for the count, not the posts. The four buckets sum to 88.
The same months with match instead of match_phrase also returned 88 hits. That run took 630 ms. The phrase time is not a floor.
When the field is not a date
A 40-line fixture stored fractional epoch strings such as 1700000000.000100 in ts. Autoindex typed ts as double and indexed the file in 0.6 s. The 20-year integer-epoch rule did not apply: the mapping shows double, not long.
curl -s -XPOST http://127.0.0.1:9217/epoch-jsonl/_search -H 'content-type: application/json' -d '{"size":0,"query":{"match_all":{}},"aggs":{"per_month":{"date_histogram":{"field":"ts","calendar_interval":"month"}}}}'
{"took":0,…,"hits":{"total":{"value":40,"relation":"eq"},…},"aggregations":{"per_month":{"buckets":[]}}}
The call succeeded. The buckets were empty. An empty list here means the field is not a date, not that the word never occurs.
Measured on 2026-10-20
| Check | Result |
|---|---|
match_phrase title autoindex, date_histogram month | 88 hits. Jul 1, Aug 50, Sep 11, Oct 26. took 123 ms |
match title autoindex, same histogram | Same four months, 88 hits. took 630 ms |
| Issue export index | 1,532 records, wall 7.2 s, exit 0. createdAt is date |
Fractional epoch fixture, ts typed double | 40 records, wall 0.6 s. Histogram buckets empty, took 0 |
One node, XERJ_MAX_PROCESS_MEMORY_MB=4096, binary v1.0.0-rc.97. A warm replay of the two issue histograms returned the same buckets with took 0. The times above are the first runs.
When not to use XERJ
If the posts already live in one table and the date column is a real date, a SQL GROUP BY month is the smaller tool. Use the histogram when the file is JSONL and you already have the index.
FAQ
Count mentions of a word over time in a posts file?
Index the file, confirm the date field in xerj autoindex map, and send one date_histogram with calendar_interval month. The capture's phrase query returned 88 hits in 123 ms.
What if the time field is a fractional epoch?
On the fixture, ts was typed double. date_histogram on it returned buckets: [] and took 0, over 40 documents. Check the type before you read the buckets.
Is a date histogram a trend model?
No. The buckets are counts of documents whose date falls in that month. There is no model and no forecast.
Phrase or a single word?
Both returned the same four months and 88 hits. match_phrase on title took 123 ms. match on title took 630 ms. 123 ms is not a floor.
Did this run use a Reddit archive?
No. The posts file was a GitHub issue export whose createdAt values are ISO strings. Do not treat these month counts as a Reddit corpus.
Evidence
- The issue export autoindex finished in 7.2 s with 1,532 records. —
benchmarks/agent-queries-2026-10-20/01-autoindex-issues.txt - The match_phrase date histogram took 123 ms and returned 88 hits. —
benchmarks/agent-queries-2026-10-20/08-histogram-phrase.json - The match date histogram took 630 ms and returned the same four months. —
benchmarks/agent-queries-2026-10-20/08-histogram.json - A string field becomes date when at least 95% of sampled values parse. —
engine/crates/xerj-autoindex/src/infer/mod.rs:470 - The fractional-epoch fixture indexed 40 records in 0.6 s and the histogram returned no buckets. —
benchmarks/agent-queries-2026-10-20/06-autoindex-epoch.txt - date_histogram on the double field ts returned buckets empty and took 0. —
benchmarks/agent-queries-2026-10-20/08-epoch-histogram-all.json