How the internals behave when you push on the knobs.

Read this first

Evidence classes stay separate

Current qualification is in progress. The exact-vector storage boundary changed to standard typed Arrow IPC record batches on 23 July 2026. Every earlier descriptor-v8 latency, CPU, RAM, disk, request, and cost row is therefore historical and cannot support the current defaults. Fresh empty-prefix runs will be promoted only after they satisfy the expanded DBpedia/Cohere/LAION/MS MARCO/filter/multitenant/hybrid/late-interaction matrix.

Full-corpus AWS results, controlled local method experiments, synthetic scale studies, direct external measurements, and vendor-reported context are never merged into one ranking. Every chart is backed by a checked-in CSV, and missing method/dataset cells remain explicitly unmeasured. The canonical reviewable index is docs/research/.

Canonical corpus

What has been evaluated

Historical corpus: Fashion-MNIST, GloVe, SIFT, NYTimes, GIST, and Deep-Image retain descriptor-v8 source recreations and archived curves for regression research. Current matrix: DBpedia OpenAI 1M, Cohere 1M/10M, LAION 100M, MS MARCO V2 138M, filters, multitenancy, BEIR hybrid, ColBERT late interaction, and mixed cache coverage are now required before publication.

• Standard datasets and recall curves
• All leaf methods and coverage gaps
• Layout, probes, candidates, width, caps, and caches
• Filtering, mixtures, updates, 1M, and 100M
• S3 Vectors, related systems, and publication position
• Reproduction and artifact validation

Parquet versus Vortex

Corrected real-segment replay keeps Parquet as the default

The first Vortex latency comparison was invalid because it stopped before materializing compressed Vortex arrays to the Arrow values consumed downstream. The corrected AWS replay covers all 17 normal-segment objects from a real Fashion-MNIST-784 index, with 3 warmups and 30 measured repetitions for every object, format, and workload. All 7,650 timed rows use the same materialized_arrow boundary.

Vortex coalesced selective native-S3 scans and won projection, point, range, and filter workloads in that historical reader. Parquet won full materialization and was much smaller: 96.84 MiB versus 253.78 MiB for Vortex default and 319.56 MiB for compact. That selective advantage was not the production query path, and the later frozen object-role qualifications rejected promotion. The unreleased backend and its dependency graph have since been removed; this section remains revision-bound research evidence.

Public ANN corpora

Archived pre-Arrow-sidecar results

Every table row in this section predates the standard typed Arrow IPC sidecar and is historical. It remains visible for auditability, but none is a current production/default claim. The replacement runs report startup separately from uncached, zero-backing-GET disk-cached, and 0/25/50/75/100% mixed-coverage p50/p95/p99/mean/stddev.

Guarantee boundary: production pq-scan reports empirical recall on each complete query set. Exact/guaranteed_recall is the formal 1.0 path and remains RAM-bounded, but it may read and score the full eligible corpus when safe lower bounds cannot prune. The NYTimes control measured 405.6 MB/query and 5.19 s disk-cached p95 for exact search; an empirical 1.000 ANN result is never relabelled as guaranteed.

Quality versus time

Recall/latency curves for every production corpus

Each point changes nprobe while keeping the dataset and engine fixed. Higher recall usually requires more cells, bytes, GETs, and exact rerank work; the selected production point is the lowest tested configuration at or above recall 0.95, not the fastest point at arbitrary quality.

Experimental graph path

From 1.95 seconds to 28 milliseconds at unchanged recall

On the same graph-enabled Fashion-MNIST index and all 100 queries, a score-once best-first frontier reduced graph p95 from 1,951.5 to 208.8 ms. Sharing the immutable decoded/validated graph in the byte-accounted cell cache reduced it to 28.5 ms. Moving graph preparation into warm() removed the first-query metadata tail: 28.0 ms p95, 28.5 ms p99, zero measured-query GETs, and the same 0.970 recall at width 512.

The recall-matched point is wider: nprobe=32, 2,560 candidates/cell reaches 0.986 recall. Across three fresh repetitions its p95 is 56.4–57.7 ms (mean 56.9), versus pq-scan at the same 0.986 recall and 89.2 ms, and Vamana-PQ at 0.988 recall and 96.2–96.4 ms. These are memory-preloaded experimental graph results on one corpus—not a replacement for the six-corpus graph-free production default.

Scale baseline · AWS eu-central-1

100M vectors stay memory-bounded; S3 latency is the next target

The first complete 100M×96D angular recreation reached 0.992 recall at 64 probes with 910.5 ms uncached and 44.4 ms disk-cached p95. Doubling probes to 128 and 256 did not change recall; it raised uncached p95 to 1.24/1.70 s and disk-cached p95 to 79/146 ms, so both rows are rejected.

The counters isolate the failure: 76.19 MB/query is the logical code span, while only 0.48 MB/query is additional uncached exact-rerank and final-record traffic. The v8 packed reader fetched from the first to the last selected slice in an object, including unrelated cells between them. A balanced 64-probe payload is about 26.56 MB before headers, so the measured span was 2.87× that lower bound. The v9 reader uses bounded, gap-aware ranges and never folds more than 64 KiB of unselected data into one code read; promotion waits for its fresh lower-probe AWS curve.

This historical result is not current evidence. The present pq-scan, srht-pq-scan, structured fast-turboquant-scan, and dense TurboQuant reference controls are rebuilt and labeled separately.

Cache-tier research · local synthetic control

Requested hot queries are not assumed to be cache hits

The cache benchmark records each query's requested class (hot or outside_hot_set) separately from its observed decoded-RAM, local-disk, and backing-storage access fractions. Shared cells and LRU eviction mean those are different facts: an outside query may reuse a hot cell, while a seeded query may miss after eviction.

On a fixed 100k-vector, 96D, 25-cell synthetic graph index, 128 MiB retained only 85% of accesses for the 100%-requested-hot mix and measured 117.3 ms steady p95. The smallest tested complete cap was 256 MiB: 100% decoded access, 0.285 ms steady p95, 1.831 ms p95 with 16 callers, and 227.3 MiB peak RSS. A 512 MiB cap did not improve that resident set. Queries outside the seeded set still paid a 112–129 ms p95 miss, so publication charts show all-query, hot-query, and outside-hot-set tails separately.

The matching auto control retained only 12/26 segment graphs at 64 MiB, reported coverage_complete=false, and served through srht-pq-scan without failing. At 256 MiB it retained 26/26, reported complete coverage, and selected the graph. These are local controls; the six-corpus AWS matrix must reproduce the envelope before any cache size becomes a default.

Whole-process telemetry

CPU, RAM, process disk, and cache growth

These production traces include the embedded engine, language/runtime overhead, query buffers, and cache—not just an artifact-size estimate. Uncapped multi-user traces remain a separate throughput-ceiling experiment. The v7 GloVe trace uses four admitted searches and the shared exact-rerank read gate.

Comparable economics

Do not charge only BORSUK for the client

Every system needs application/client compute. BORSUK runs search inside that process; S3 Vectors, turbopuffer, Pinecone, and Chroma receive the request from it and include remote search compute in their service price. The headline product rows therefore exclude common client compute for every system. BORSUK reports its selected S3 index storage and measured GETs, while CPU/RAM/disk stay explicit in the resource evidence above. See the dated cost formulas and current vendor list prices.

More

Reproduce and extend

Each sweep has an ignored gate test that regenerates its CSV — see the section copy for the exact test name. Full-corpus, method, configuration, scale, comparison, and reproduction notes are in the canonical docs/research/ hierarchy. Raw artifacts remain under docs/web/assets/benchmarks/.