--- title: "Batch lookup of CVE and GHSA IDs offline, with sources" canonical: "https://xerj.org/answers/batch-lookup-cve-ghsa-ids-offline" updated: "2026-10-20" source: "content/answers/batch-lookup-cve-ghsa-ids-offline.md" --- # Batch lookup CVE IDs and GHSA IDs offline, without GitHub API rate limits **TL;DR** — To batch lookup CVE and GHSA IDs offline without GitHub API rate limits, load the signed rust-vulns advisory pack into xerj and resolve every ID in one `terms` query. Each answer names the source file; unknown IDs abstain. On 2026-10-19, 6 IDs took 15 ms: 4 resolved, 2 abstained. Coverage is Rust crates only. ## Commands ### Command 1 Note: Start a single-user node on localhost:9200 in its own terminal. ```sh xerj --insecure --data-dir ./xerj-data ``` ### Command 2 Note: Install the advisory pack after checking its signature. ```sh xerj corpus add rust-vulns --from rust-vulns-pack.zip --verify-sig rust-vulns.pub ``` ### Command 3 Note: Index the RustSec and OSV records into xc-rust-vulns-* indices. ```sh xerj corpus index rust-vulns ``` ### Command 4 Note: Resolve every ID in ids.txt with one _search; print the record and source, or abstain. ```sh bash batch.sh ``` ## Run it Download the zip, the public key and the signature from the GitHub release `pack-rust-vulns-2026-10-19`. Then install the pack with a signature check. ```sh xerj --insecure --data-dir ./xerj-data & xerj corpus add rust-vulns --from rust-vulns-pack.zip --verify-sig rust-vulns.pub xerj corpus index rust-vulns ``` The add step printed `signature verified` in about 1.3 s. It loaded 1,288 RustSec and 745 OSV advisories. Indexing the 2,033 advisories took 9,769 ms. Put the IDs in a file: ```text CVE-2021-25900 GHSA-9q5j-jm53-v7vr CVE-2026-35355 RUSTSEC-2023-0071 CVE-2024-24576 CVE-2099-00001 ``` `batch.sh` sends all of them in one request and matches each ID against `id` or `aliases`: ```sh ids=$(jq -R . ids.txt | jq -sc .) curl -s 'localhost:9200/xc-rust-vulns-*/_search' -H 'content-type: application/json' -d "{ \"size\": 100, \"_source\": [\"id\", \"aliases\", \"packages\", \"fixed_versions\", \"origin\", \"source_url\"], \"query\": {\"bool\": {\"should\": [{\"terms\": {\"id\": $ids}}, {\"terms\": {\"aliases\": $ids}}]}} }" | jq -r --argjson ids "$ids" ' [.hits.hits[]._source] as $h | $ids[] as $q | ($h | map(select(.id == $q or ((.aliases // []) | index($q)))) | first) as $r | if $r then "\($q) \($r.id) \(($r.packages // []) | join(",")) fixed \(($r.fixed_versions // []) | join(",")) src \($r.source_url) :: \($r.origin)" else "\($q) NOT FOUND in this pack: abstain" end' ``` Real output, 15 ms wall time: ```text CVE-2021-25900 RUSTSEC-2021-0003 smallvec fixed 0.6.14,1.6.1 src https://github.com/rustsec/advisory-db :: crates/smallvec/RUSTSEC-2021-0003.md GHSA-9q5j-jm53-v7vr RUSTSEC-2022-0051 lz4-sys fixed 1.9.4 src https://github.com/rustsec/advisory-db :: crates/lz4-sys/RUSTSEC-2022-0051.md CVE-2026-35355 GHSA-239g-2685-54x3 uu_install fixed 0.6.0 src https://osv-vulnerabilities.storage.googleapis.com/crates.io/all.zip :: GHSA-239g-2685-54x3.json RUSTSEC-2023-0071 RUSTSEC-2023-0071 rsa fixed src https://github.com/rustsec/advisory-db :: crates/rsa/RUSTSEC-2023-0071.md CVE-2024-24576 NOT FOUND in this pack: abstain CVE-2099-00001 NOT FOUND in this pack: abstain ``` RUSTSEC-2023-0071 shows no fixed version because the advisory lists none. A trailing wildcard (`xc-rust-vulns-*`) is the pattern that worked; see the last section for the two that did not. ## Measured on 2026-10-19 | Measure | Value | | --- | --- | | Pack | rust-vulns, 1,598,960 bytes, 2,033 advisories, built 2026-10-19 | | Signature check and load | about 1.3 s | | Index | 9,769 ms | | Batch of 6 IDs, one request | 15 ms wall time | | Resolved / abstained | 4 / 2 | | GitHub API calls after download | 0 | ## How it compares | Approach | Offline | Rate limit | Many IDs per call | Source per answer | Coverage | | --- | --- | --- | --- | --- | --- | | xerj + rust-vulns pack | yes | none, local | yes, one `terms` query | yes, file in advisory-db or OSV | Rust crates | | GitHub advisory REST API | no | yes, per token | no, one `cve_id` per request | yes | GitHub ecosystems | | OSV.dev API | no | service limits | yes, `querybatch` | yes | many ecosystems | | `cargo audit` | after DB fetch | none | per lockfile, not per ID | yes | Rust crates | | Chat model | depends | depends | yes | no, may invent an ID | training data | ## When not to use xerj - Your IDs are not Rust crates. The pack holds RustSec and OSV crates.io advisories only; npm, PyPI and Go are not in it. - You need advisories published since the build date. The pack was built 2026-10-19 and does not update itself. - You want to audit a `Cargo.lock`. `cargo audit` does that directly. - You need `_msearch` or a mid-name wildcard. On rc.97, `_msearch` with `xc-rust-vulns-*` returned 404 index_not_found, and `_search` on `xc-rust-vulns-*-rustsec` returned 0 hits without an error. Use the trailing wildcard as above. ## FAQ ### How do I batch lookup CVE IDs without the GitHub API? Download the signed rust-vulns pack once, load it with `xerj corpus add` and `xerj corpus index`, then send all IDs in one `terms` query on `id` and `aliases`. Nothing calls the GitHub API after the download. ### How do I avoid GitHub API rate limits when looking up advisories? Query a local copy. The pack is a 1,598,960-byte release asset; after that every lookup is a request to your own node on localhost. ### Can I search vulnerability advisories offline? Yes, for the advisories in the pack: 2,033 records from RustSec (1,288) and OSV crates.io (745), built 2026-10-19. It covers Rust crates only. ### How do I cite the source for a CVE's affected versions? Each record carries `source_url` and `origin`, for example `crates/smallvec/RUSTSEC-2021-0003.md` in rustsec/advisory-db. Print them next to the fixed versions, as the script on this page does. ### What happens to an ID that is not in the pack? The script prints `NOT FOUND in this pack: abstain`. CVE-2024-24576, a Rust standard library CVE, was not in this crate-advisory pack and abstained. ### Is there a built-in batch command? Not yet. Many IDs per call with a per-item answer or abstain is issue #1801, open. Today the batch is one `_search` plus jq. ### Can I use xerj code with a CVE ID instead? Not reliably. `xerj code rust-vulns "CVE-2021-25900"` ranked an lz4 advisory first in our run. Exact ID matching in `xerj code` is issue #1788; use the `terms` query. ## Evidence - Six IDs: four resolved, two abstained — `benchmarks/agent-intents-2026-10-19/05-batch-output.txt` - The batch script — `benchmarks/agent-intents-2026-10-19/05-batch.sh` - xerj code ranked the lz4 advisory first for CVE-2021-25900 — `benchmarks/agent-intents-2026-10-19/05-xerj-code-exact-id.txt` - Pack load about 1.3 s, index 9,769 ms, batch 15 ms — `benchmarks/agent-intents-2026-10-19/README.md` - Batch queries are open as issue #1801 — [https://github.com/xerj-org/xerj/issues/1801](https://github.com/xerj-org/xerj/issues/1801) ## Related - [Full-text search over JSONL files from the CLI: index a folder and query it with BM25](/answers/full-text-search-over-jsonl-files-cli) - [Retrieval that says not found, and cites file and line when it does answer](/answers/retrieval-that-says-not-found-with-file-line-citations) - [One vulnerability, three names](/blog/one-vulnerability-three-names) - [The exploit hub](/blog/the-exploit-hub) - [Is there an Elasticsearch-compatible engine that isn't Elasticsearch?](/compare/xerj-vs-elasticsearch-local)