ONE VULNERABILITY, THREE NAMES,
AND A PACK WE SIGN
The same Rust vulnerability arrives as RUSTSEC-2021-0003,
GHSA-43w2-9j62-hq99 and CVE-2021-25900 — three
records, in two databases that never talk to each other, none of them carrying everything the other
knows. We built a tool that collapses them into one record, packs it into something portable and
checksummed, and then signs the pack. The last part is the strangest finding of the project: in our
survey of vulnerability databases people pip-install and OCI-pull every day, not one ships a
signature a consumer can verify.
Two sources, each sure it is the whole story
We started with the sources everyone reaches for, and measured what each one actually carries for the Rust ecosystem:
| Source (both CC0) | Records | Has | Missing |
|---|---|---|---|
| osv.dev crates.io export | 2,856 | GHSA + RUSTSEC + MAL ids, CVSS scores | zero affected-function data; aliases never merged |
| RustSec advisory-db | 1,251 | curated vulnerable-function paths, categories, keywords, full advisory prose | no GHSA-only records |
| GitHub advisory-database | — | the same crates.io records | its extra fields are marked internal-use-only — we never read them |
The alias problem is the one that hurts an agent most. osv.dev's export lists the same vulnerability twice — once under its GHSA id, once under its RUSTSEC id — with a shared CVE in each record's alias list, and no merge ever performed between them. Search it for “smallvec insert_many” and you get whichever filing happened to phrase it that way; the other half of the knowledge is a separate record you did not retrieve. RustSec fixes the merge by being one curated database, but it cannot see GHSA-only filings at all.
HackerOne, the source the original brief asked about, is out of scope for a different reason: report content there is unredistributable under its terms of service. A pack's author owns what goes in the pack, and this pack ships only sources whose licence lets us redistribute them.
Identity resolution as a declarative file
We did not write a Rust-vulnerability importer. We wrote a corpus builder whose entire domain knowledge lives in a commented TOML recipe — because the next corpus is not advisories. The same tool with a different recipe indexes mailboxes, or code, or genomes; nothing about the pipeline knows what a vulnerability is.
The identity section is three lines:
[identity]
edges = [{ field = "id" }, { field = "aliases", each = true }]
canonical_source_order = ["rustsec", "osv"]
Union-find over those edge values does the rest, and it closes chains for free: a RustSec filing and a GHSA filing that share a CVE alias merge even with no direct edge between them, because each is transitively bound to the CVE. Measured on the real sources:
| rust-vulns, measured off the committed build | Count |
|---|---|
| source records harvested (envelopes) | 4,107 |
| records after identity resolution | 1,950 |
| …merged from both sources | 1,221 |
| canonical id: RUSTSEC / GHSA / MAL | 1,221 / 711 / 18 |
| records carrying curated function paths | 270 (663 distinct paths) |
| withdrawn advisories (kept, flagged) | 124 |
| packages covered · records JSONL bytes · cold build | 1,813 · 10,140,789 · ~34 s |
The function paths are the field no export carries. RustSec's hand-curated
affected.functions — which function in which crate is the vulnerable
one — survives the merge unioned across sibling filings: one 2026 CVE filed against three
libcrux-* crates becomes one record carrying all three crates'
paths. For an agent asking “is this call the dangerous one?”, that is the
whole difference between a hit and an answer.
Merge precedence is declared, not discovered: RustSec wins a disagreeing scalar because it
owns the curated fields; arrays are always unioned; the merged record's licence is the most
restrictive of its sources and sources[] names every contributor,
so a merge is auditable after the fact. Withdrawn advisories are kept and flagged rather than
dropped — filtering is the consumer's decision, not the pack's.
A directory you can move, and a machine that refuses surprises
The build emits a self-contained directory: sharded records in canonical JSON order, a manifest with per-file SHA-256 checksums, the recipe verbatim as provenance, and suggestions — a proposed field mapping and join candidates computed from a sample, explicitly suggestions, not decisions, because the index infers the real mapping the way it does for any folder.
Two rules in the format are load-bearing. A manifest carries a
format_version that readers refuse when they do not know it — a
published pack outlives the release that built it. And installing one verifies every checksum
before anything is materialized, so a truncated or edited pack is refused whole, not half-mounted:
$ xerj corpus add rust-vulns --from rust-vulns-pack.zip --verify-sig rust-vulns.pub $ xerj corpus index rust-vulns $ xerj code rust-vulns "smallvec insert_many buffer overflow"
Incremental rebuilds need no mode flag: every normalized record is stored content-addressed, so a re-run recomputes each key, sees the file, and skips. Only new or changed upstream records are written, and the pack re-emits deterministically from whatever the store holds — same bytes in, same pack out, which is what makes a signature over it meaningful.
Checksums prove integrity. They do not prove origin.
Here is the attack that checksums cannot stop. Take a pack. Edit records — flip a
fixed_versions string, say. Recompute every file's checksum and
rewrite SHA256SUMS honestly. The pack now verifies perfectly against
its own manifest. Every checksum agrees, because the checksums were never the thing being
attacked — the pack's origin was.
We pinned exactly this in a test: a self-consistent rebuild passes every checksum check and
fails the signature. That gap is the entire reason the pack carries one — a raw 64-byte
ed25519 signature over SHA256SUMS, verified by
xerj corpus add --verify-sig <pubkey> before a byte is
materialized.
Before building it we surveyed what the existing ecosystem ships, and found the lane empty. None of these is a criticism of the projects — their data is good and their tooling is real; it is a statement about the format:
| Database (our survey, 2026-09) | Distribution | Integrity mechanism |
|---|---|---|
| Trivy DB | OCI artifact, 6-hour builds | checksums in registry metadata — no consumer-verifiable signature |
| Grype / vunnel | per-provider feeds | unsigned |
| PrimeVul | Hugging Face dataset | unsigned |
| VulnRAG | research artifact | unsigned |
The design rules we landed on, each one because its absence would make the signature decorative:
- The public key travels out of band — committed beside the recipe in the repository, mirrored on our site, never inside the pack. A key shipped beside its own signature verifies nothing: an attacker who can replace the pack can replace the key.
- Build and publish are different steps.
corpus buildemits; it never signs. The seed exists only as a CI secret, and the publish workflow signs, then verifies its own output against the committed public key before publishing — a rotated key that forgot to update the committed.pubfails the build instead of shipping a pack no consumer can verify. - The signature covers the checksum file, which covers every pack file; the signature itself is deliberately absent from the checksum list, because signing a list that contains the signature is circular.
.pub, and consumers choosing to trust the new one. We wrote the
procedure down before we needed it.
What this is not
Not a security product. It is a searchable corpus of advisories for agents and humans, with no completeness claim beyond what its two sources carried at build time. It is not a scanner and will not tell you what is running in your production.
Freshness is a cron job, not a promise. The publish workflow rebuilds daily; between a source updating and the next run, the pack lags by up to a day. The README badge reads the build date off the release itself, and the same 30-day staleness refusal that governs every reference corpus governs ours — a workflow that stops running shows up as a stale pack, not a silently aging artifact.
Records, not an index. Installing the pack still costs every consumer the same CPU-minutes to index — the pre-indexed half of the corpus-hub idea is a separate, harder format problem (a published index needs a compatibility promise our internal segment format has never had to make).
The value hypothesis is unproven. Trivy's database gets pulled because scanners need it at scan time; an agent-facing vulnerability corpus has weaker daily pull, and the honest analogy is template packs that took years to compound. What we can say is what the pack uniquely carries — identity-resolved clusters, the curated function paths, a signature — and that the numbers above are measured, not projected.
Every number, pinned to a run
The counts in section 02 come from
tools/packs/rust-vulns
— a committed pack-stats.json whose provenance field says
measured, with a test that fails the build if the README's table and
the stats file drift apart, and another that fails if the stats describe any recipe other than the one
shipped. The builder, the signature code, and the pack format are in the same repository under
engine/crates/xerj-autoindex/src/harvest/, and the format is
documented end to end in
docs/CORPUS_PACKS.md.
The pack itself is a rolling GitHub Release — pack-rust-vulns — rebuilt and signed on a schedule, with the zip, the loose checksums, the signature, and a freshness file as assets. Build it yourself instead, from the recipe, if you would rather trust your own machine than our key; that is a supported path, not a workaround. If your agent works with Rust dependencies, point it at the pack and ask it the question a raw export cannot answer: which function is the vulnerable one?