Config TOML

XERJ reads one TOML file. Every key has a production-ready default, so the smallest working config is an empty file. The table below is the full surface; the sections below it walk each group with a runnable example.

Paths for the config are, in precedence order: --config /path/to/xerj.toml on the command line, then /etc/xerj/xerj.toml, then ./xerj.toml in the working directory. If none exist, the full default is used.

Full key table

[server]

Network listeners and the data directory. Most deployments only touch data_dir and the bind address.

[server]
rest_port      = 8080            # native /v1/* API
es_compat_port = 9200            # ES wire-compatible API
grpc_port      = 8081            # reserved
bind_address   = "0.0.0.0"       # not a default — the default is "127.0.0.1" (loopback only)
allow_insecure_network_bind = true   # not a default (false) — required for a non-loopback bind while TLS is off
data_dir       = "/var/lib/xerj" # not a default ("./data") — an absolute path is strongly recommended

The default bind is loopback, so an unconfigured node is not reachable from the network. Exposing it while tls.enabled = false refuses to start unless allow_insecure_network_bind says the cleartext exposure is intended — enable TLS and neither line is needed.

[auth]

Static API-key authentication. The first-run admin key is written to <data_dir>/admin.key; subsequent starts reuse it. Clients pass Authorization: ApiKey <key> on every request.

[auth]
enabled       = true
admin_api_key = ""               # blank → auto-generated on first run

# Or provide your own:
# admin_api_key = "ak_live_c8f9a4…"

[tls]

TLS termination at the server. In Kubernetes or behind a load balancer, leave this off and terminate at the proxy instead — one place to rotate certs, one place to log handshakes.

[tls]
enabled   = true                         # not a default (false) — this block switches TLS on
cert_path = "/etc/xerj/certs/server.crt" # not a default ("") — your certificate
key_path  = "/etc/xerj/certs/server.key" # not a default ("") — your private key

[storage]

The WAL and flush tuning. wal_sync is the durability knob everyone looks for — pick "sync" for financial/compliance workloads, "batched" for everything else, and "async" only in benchmarks.

[storage]
wal_sync            = "batched"
wal_batch_ms        = 100        # fsync every 100 ms
wal_max_size_mb     = 1024       # roll WAL every 1 GiB
flush_size_mb       = 512        # flush memtable at 512 MiB
flush_interval_secs = 30

[merge]

Segment compaction. size_tiered is the only implemented policy: it merges same-size segments, which is cheap and write-optimal. log_structured is a name in the enum with no merge policy behind it, so the server refuses to start on it rather than silently running size-tiered merging for an operator who picked a levelled policy for its read amplification. If full-range scans are what hurts, that is a real gap in this build — setting the key will not change it.

[merge]
strategy           = "size_tiered" # the only one implemented — log_structured is refused at startup
min_segments       = 10          # accepted, not wired
max_segment_mb     = 8192        # 8 GiB cap
io_rate_mb_per_sec = 100         # accepted, not wired — merges are not throttled
max_concurrent     = 1           # accepted, not wired (see XERJ_MERGE_PARALLELISM)
tier_floor_mb      = 4
min_merge_count    = 4           # the real per-tier merge trigger
max_merge_count    = 16          # caps peak merge RAM

[compression]

See Compression for the encoding catalog. This section picks the outer block codec only — the inner per-column encodings are chosen automatically at write time.

[compression]
enabled         = true
level           = "balanced"     # LZ4 / Zstd L3 / Zstd L19
block_size_docs = 128

[fts]

Default analyzer applied to untyped text fields. Override per-field in the mapping when creating an index. See Analyzers for the built-ins.

[fts]
default_analyzer = "standard"    # unicode words + lowercase

[vector]

Vector search settings. Unfiltered kNN is served by a persisted HNSW graph with exact rescoring (measured recall@10 1.00 on the official bench query); filtered kNN and SQ8 fields run the exact scan. The hnsw_* keys are accepted and validated but not yet wired: the graph builds with fixed M=16 / ef_construction=200, and the query-time beam width comes from the request's num_candidates (floored at 800). Quantization is what actually trades memory for recall.

[vector]
default_metric         = "cosine"
hnsw_m                 = 16          # accepted for compat; kNN serving is exact
hnsw_ef_construction   = 200         # accepted for compat; no effect on results
hnsw_ef_search         = 100         # accepted for compat; no effect on results
default_quantization   = "none"      # "scalar8" = 4× RAM saving, 1–2% recall loss
max_dimensions         = 16384

[logs]

Time-series retention. Log indices are sliced into partitions of time_partition width so retention prunes are O(partitions), not O(documents).

[logs]
retention_days = 30              # not a default (90) — keep 30 days
time_partition = "1h"            # 1-hour partitions

[embedding]

Delegates vector generation to an OpenAI-compatible endpoint. Leave default_endpoint empty if clients provide vectors themselves. Token limits are model-specific; the chunker in the ai crate splits long documents to fit the model's window.

[embedding]
# OpenAI:
default_endpoint = "https://api.openai.com/v1/embeddings"  # not a default ("" disables auto-embedding)
default_model    = "text-embedding-3-small"                # not a default ("")
batch_size       = 64
timeout_ms       = 5000

# Or a local Ollama:
# default_endpoint = "http://localhost:11434/v1/embeddings"
# default_model    = "nomic-embed-text"

[limits]

Hard caps to protect the server from runaway queries and mapping explosions. Lower these on shared nodes, raise max_query_memory_mb for aggregation-heavy workloads.

[limits]
max_query_memory_mb     = 512
max_concurrent_searches = 64
max_fields_per_index    = 500

[indexing]

Turbo mode knobs. Turbo is opt-in per-request via POST /v1/indices/:name/turbo-ingest or the X-Turbo: true header on _bulk; these settings only apply when turbo is active.

[indexing]
turbo_batch_size    = 2000       # not a default (1000)
turbo_parallel      = true
turbo_fast_analyzer = false      # true only if recall doesn't matter

[cluster]

Multi-node mode. Default is off — single-node doesn't need a consensus layer. When enabled, the embedded Raft implementation replicates metadata only (schemas, shard assignments, node roster). See Clustering for the full story.

[cluster]
enabled = true                   # not a default (false) — this block switches cluster mode on
port    = 9300                   # intra-cluster gRPC + Raft
peers   = [                      # not a default ([]) — your node roster
  "a=10.0.0.11:9300",
  "b=10.0.0.12:9300",
  "c=10.0.0.13:9300",
]
tick_ms = 50

Source · engine/xerj.default.toml · engine/crates/xerj-common/src/config.rs