What product quantization cost
57.7% recall@10 at 16x compression for flat PQ, and 99.3% once the top 100 are re-ranked.
Product quantization (PQ) trades recall for memory. Instead of storing full-precision vectors, it trains k-means++ codebooks per subspace and stores compressed codes. NuclaDB's implementation (internal/index/pq) uses asymmetric distance computation (ADC), so the query vector itself is never quantized, only the stored vectors are.
The number
57.7% recall@10 at a fixed 16x memory reduction (M=16, Dim=64), from internal/index/pq/pq_test.go's TestIndexRecallVsExact, which is deterministic across 5 runs with fixed seeds.
This is flat, unindexed PQ, on purpose
No re-ranking, no IVF coarse quantizer. Production PQ setups typically add both, and so does NuclaDB now (below). The 57.7% leaves them out deliberately, as a clean measurement of quantization error alone.
Re-ranking and IVF, measured
Both follow-ups now exist as library code in internal/index/pq.
Re-ranking (pq.Index.SearchRerank) takes the top candidates by PQ distance and re-scores them against their full-precision vectors. Same data and codebooks as the 57.7% number:
| Configuration | recall@10 |
|---|---|
| PQ alone | 0.577 |
| PQ + re-rank top 50 | 0.960 |
| PQ + re-rank top 100 | 0.993 |
| PQ + re-rank top 200 | 1.000 |
Re-ranking needs the full vectors, so it gives back the memory saving: it fits when codes live in RAM and full vectors live on disk.
IVF (pq.IVFIndex) puts an inverted file in front of the codes: k-means splits the vectors into nlist lists, and a search scans only the nprobe lists closest to the query. Uniform random data has no cluster structure for IVF to use, so this was measured on 5,000 clustered vectors, 30 held-out queries, nlist = 32, re-ranking the top 100:
| nprobe | share of codes scanned | recall@10 |
|---|---|---|
| 1 | ~1/32 | 0.637 |
| 4 | ~1/8 | 0.870 |
| 8 | ~1/4 | 0.943 |
| 32 | all | 0.993 |
Not in the server yet
PQ, re-ranking and IVF are tested packages. nucladbd doesn't use them: every tenant is a full-precision HNSW index.
What's verified
The distance computation itself is verified exact, so the recall loss traces to quantization error alone, not a scoring bug. That distinction matters: a lower number from a correct-but-lossy algorithm is one kind of tradeoff, while a lower number from a buggy comparator would just be a bug.
For the other side of the compression-vs-accuracy space, see HNSW ef tuning.
Tuning HNSW: what the recall/latency curve looks like
The full recall@10/QPS sweep on SIFT-small, and why efSearch is a per-query parameter.
What Raft gave the system and what it cost
Agreed-upon topology through faults, proven under chaos, and the linearizability window that async replication leaves open.