--- title: "Look up many IDs in one local query" canonical: "https://xerj.org/answers/batch-lookup-ids-from-csv-against-local-index" updated: "2026-10-20" source: "content/answers/batch-lookup-ids-from-csv-against-local-index.md" --- # Look up 1,000 IDs against a local index **Short answer** (updated 2026-10-20): Look up 1,000 IDs against a local index with one `terms` query, then mark each ID found or not found. On 2026-10-20 the query sent 1,000 numbers over 532 issues. The response `took` was 0. 532 IDs were in the index and 468 were not. **Key number:** A terms query of 1,000 IDs against 532 issues returned 532 hits and took 0 ms. (as of 2026-10-20; source: [benchmarks/agent-queries-2026-10-20/08-batch-terms.json](https://github.com/xerj-org/xerj/blob/main/benchmarks/agent-queries-2026-10-20/08-batch-terms.json)) **TL;DR** — Look up 1,000 IDs against a local index with one `terms` query, then mark each ID found or not found. On 2026-10-20 the query sent 1,000 numbers over 532 issues. The response `took` was 0. 532 IDs were in the index and 468 were not. ## Agent prompt ```text Read https://xerj.org/llms.txt. Start one local node, index the dataset, and read the id field type from xerj autoindex map. Send one terms query with every id from the CSV. Print found or not found for each id from the hits. A wildcard index name on _msearch is not a pattern. ``` ## Commands ### Command 1 Note: The shape of the query. The capture sent 1,000 numbers, not these three. ```sh curl -s -XPOST http://127.0.0.1:9217/gh-jsonl/_search -H 'content-type: application/json' -d '{"size":0,"track_total_hits":true,"query":{"terms":{"number":[960,488,484]}}}' ``` ### Command 2 Note: Named index gh-jsonl. The capture's item status was 200. ```sh curl -s -XPOST http://127.0.0.1:9217/_msearch -H 'content-type: application/x-ndjson' --data-binary @benchmarks/agent-queries-2026-10-20/fixtures/msearch-named.ndjson ``` ### Command 3 Note: One misspelling. Issue 960 ranked first, took 1 ms. ```sh curl -s -XPOST http://127.0.0.1:9217/gh-jsonl/_search -H 'content-type: application/json' -d '{"size":3,"_source":["number","title","state"],"query":{"match":{"title":{"query":"feedbak open-pr","fuzziness":"AUTO"}}}}' ``` XERJ is a new way for AI to search for data. ## One terms query The id field on the issue export was `number`, type `long`. The capture built 1,000 integers: every issue number in the export, then unused integers up to 1,000. The list is `04-ids.txt`. ```sh curl -s -XPOST http://127.0.0.1:9217/gh-jsonl/_search -H 'content-type: application/json' -d '{"size":0,"track_total_hits":true,"query":{"terms":{"number":[960,488,484]}}}' ``` The command above shows the shape with three ids. The captured body sent all 1,000. `size` 0 asks for the count. The response was `took` 0 and `hits.total.value` 532. The other 468 ids were absent. Found or not found is that comparison. It is not a per-id abstain feature. Guessing the index name `gh-issues` returned HTTP 404. The issues lived in `gh-jsonl`. ## _msearch needs the real name A wildcard header did not expand: ```text {"index":"gh-*"} ``` The HTTP status of the call was 200. The item inside was status 404, `no such index [gh-*]`. The same call with `{"index":"gh-jsonl"}` and 6 issue numbers returned item status 200, `took` 0, and 6 hits. ## One fuzzy query, nothing more ```sh curl -s -XPOST http://127.0.0.1:9217/gh-jsonl/_search -H 'content-type: application/json' -d '{"size":3,"_source":["number","title","state"],"query":{"match":{"title":{"query":"feedbak open-pr","fuzziness":"AUTO"}}}}' ``` `took` was 1. 13 hits. Issue 960 was first, then 488, then 1837. That is the whole observation. ## Measured on 2026-10-20 | Check | Result | |---|---| | 1,000 ids, `terms` on `gh-jsonl` | `took` 0. 532 found, 468 absent | | `_msearch` index `gh-*` | Item status 404, `no such index [gh-*]` | | `_msearch` index `gh-jsonl`, 6 ids | Item status 200, 6 found, `took` 0 | | `fuzziness` `AUTO` on `feedbak open-pr` | Issue 960 first, 13 hits, `took` 1 ms | | Index name `gh-issues` | HTTP 404 | ## When not to use XERJ If the ids are exact and they already live in one table, a SQL join is the smaller tool. Use the terms query when the ids are a file and the records are an index you already built. ## FAQ ### Look up 1,000 IDs against a local dataset? One `terms` query on the id field. The capture sent 1,000 numbers to `gh-jsonl`. `took` was 0. 532 were issues in the index and 468 were not. ### Batch query a local index from a CSV of IDs? Turn the column into a JSON array and put it in `terms`. Print found or not found yourself. There is no batch subcommand. ### msearch several queries in one request? Name the index. `{"index":"gh-*"}` came back as item status 404, `no such index [gh-*]`. `{"index":"gh-jsonl"}` came back status 200. ### Join a CSV with a fuzzy match? One observed misspelling, `feedbak open-pr`, ranked issue 960 first in 1 ms, with 13 hits. That is one query, not a measured join accuracy. ### What if the index name is wrong? Guessing `gh-issues` returned 404. The real names were `gh-jsonl` and `gh-json`. Read `xerj autoindex map`. ## Evidence - A fuzziness AUTO query took 1 ms and ranked issue 960 first. — `benchmarks/agent-queries-2026-10-20/08-fuzzy.json` - The terms query over 1,000 ids returned took 0 and 532 hits. — `benchmarks/agent-queries-2026-10-20/08-batch-terms.json` ## Related - [I have a directory of CSV exports. How do I query them without opening each one in Excel?](/answers/index-multiple-csv-files) - [What's the easiest way to search JSON logs plus some old gzip text logs in the same folder?](/answers/search-json-and-jsonl-logs) - [Find duplicate GitHub issues before you file](/answers/find-duplicate-github-issues-before-filing)