Ninety Six Bytes Each

You have 40 million 768-dimension float32 embeddings and 16 GB of RAM on the search node. You apply product quantization: split each vector into $m = 96$ contiguous subvectors of 8 dimensions, and replace each subvector with the index of its nearest centroid in a per-subspace codebook of 256 centroids.

What is the per-vector code size, what does the whole set occupy, and what must you add to keep answer quality?

8 bytes per vector, 320 MB for the set, since each subvector collapses to a single 8-bit code and there are 8 dimensions per subvector.

96 bytes per vector, 3.84 GB for the set. PQ distances are computed from centroids and carry error, so you keep the full-precision vectors on disk and re-rank the PQ shortlist against them.

96 bytes per vector, 3.84 GB for the set. No re-ranking is needed: the codebook is trained on your own data, so the reconstructed distances are exact.

768 bytes per vector, 30.7 GB for the set, which still does not fit, so you need a second node.