Can you store embeddings as int8?

Yes — int8 scalar quantisation stored 100,000 vectors in 3.96x less space and still returned 0.984 of the float32 top-10, rising to 1.000 once the top 100 candidates were re-scored with float32. The surprise is what embedding quantization did not buy: the int8 scan was no faster than float32, and float16 was measurably slower. Bytes and time are separate budgets, and only one of them improves.

Everything below uses randomly generated vectors, and that matters less here than you would think. The previous article in this series, do you actually need a vector database, had to disclaim every recall number it produced, because it compared an approximate index against ground truth and random data has no cluster structure for a graph index to exploit. This article measures something different: the same vectors stored two ways, compared against themselves. That is quantisation error, not semantic quality. If int8 reorders the neighbours of a vector, it reorders them whatever the vector means. The float32 baseline is computed from the identical corpus, so the comparison is internally valid and the control row reads 1.000 in every table below.

The honest limitation runs one way only. Real embeddings are not uniformly distributed — they cluster, so their true nearest neighbours sit further from the rest of the field than random neighbours do, and a bigger gap between the 10th and 11th result is a bigger error budget for the quantiser to eat. So recall on real embeddings is normally better than these numbers, not worse. Read them as a conservative floor, not a prediction of what your corpus will do.

Hardware: Apple M3, 16 GB, macOS 26.4.1. Node v23.5.0, Python 3.14.6, no numpy, no network, no model call. All timings are indicative of one laptop under normal desktop load, single core, in process, vectors already resident; spread columns are shown because that load is visible in the results.

The short answer

• int8 scalar quantisation is 3.96x smaller than float32 at 384 dimensions (388 bytes per vector against 1,536, including the per-vector scale factor) and 3.99x smaller at 1,536 dimensions.
• int8 kept 0.984 of the float32 top-10 at 384 dimensions and 0.977 at 1,536, on uniformly random vectors — the hardest case.
• Re-ranking the top 100 int8 candidates with float32 restored recall@10 to 1.000 at both dimensions, at a cost of re-reading 100 float32 vectors: 0.15 MB against a 153.6 MB full scan.
• Smaller vectors did not scan faster. int8 took 45.1 ms against float32's 46.5 ms over 100,000 × 384 — inside the noise, despite reading a quarter of the bytes. float16 was 36% slower than float32.
• Binary quantisation is the exception in both directions: 32x smaller and 13.7x faster, but recall@10 of 0.062, and still only 0.931 after re-ranking 10,000 candidates — a tenth of the entire corpus.

How much smaller is int8?

Vectors are L2-normalised float32, so cosine similarity is a plain dot product. int8 uses per-vector symmetric max-abs quantisation — the standard scheme: divide every component by the vector's largest absolute value, scale to ±127, round, and store the scale as one float32 alongside. Binary keeps one sign bit per dimension. Node 23.5 ships V8 12.9 and therefore has no Float16Array, so float16 is stored as a Uint16Array with a hand-written IEEE 754 binary16 codec, verified bit-for-bit against Python's struct format 'e' on 42,000 values — 0 mismatches in both directions.

Storage for 100,000 vectors, measured as byteLength of the actual buffers:

Nothing surprising, and that is the point — storage is a multiplication you can budget in advance. The only wrinkle is the 4-byte scale factor per vector, which costs 1.04% at 384 dimensions and 0.26% at 1,536. int8 is "4x smaller" everywhere except in a spreadsheet.

Does a smaller vector scan faster?

This is where the expectation broke. A scan over 100,000 vectors, median of nine runs, including top-10 selection:

int8 reads a quarter of the bytes and finishes in the same time. float16 reads half the bytes and takes 36% longer at 384 dimensions, because every element has to be turned back into a float through a 256 KB lookup table before it can be multiplied. Only binary is dramatically faster: 13.7x at 384 dimensions and 22.1x at 1,536.

Why isn't int8 faster?

Because a scalar loop costs per element, not per byte. The inner loop runs DIM iterations whether each iteration loads one byte or four, and V8 emits a load-convert-multiply-add either way. Binary wins because it is the only representation that reduces the iteration count: 12 Uint32 words instead of 384 multiplications, with a SWAR popcount over each XOR.

The obvious objection is that the corpus was small enough to hide a memory bottleneck, and int8 would pull ahead once the working set stopped fitting. It did not. Sweeping row counts at 384 dimensions with one representation per process, nanoseconds per dimension, two clean passes:

float32 and int8 track each other up to 3 GB. One earlier pass did show int8 winning at 2,000,000 rows — 649 ms against 1,201 ms — and it was an artefact: that float32 run followed two other multi-gigabyte processes in the same shell loop, and three repeat runs on a settled machine returned 651, 680 and 774 ms. It went in the bin. The lesson is that quantisation is a memory optimisation that becomes a speed optimisation only when something else makes you bandwidth-bound — a SIMD kernel that processes 16 int8 lanes per instruction, or a working set that no longer fits in RAM. In a plain JIT loop, you get the space back and nothing else.

Does embedding quantization change the ranking?

For each of 100 query vectors, the top 10 by exact float32 dot product is the ground truth. Every quantised representation is scored against that set. The float32 row is the control and must read 1.000.

float16 is lossless for ranking purposes — not a single one of the 1,000 gold results moved at either dimension. That is the cheapest 2x in the table and it would be the default if it were not slower to scan.

The mechanism is measurable. Averaged over the 100 queries at 384 dimensions, the gap in cosine score between the 10th and 11th true neighbour is 0.001258. The mean absolute error a representation introduces into a score is 0.0000085 for float16 and 0.000415 for int8. So float16's error is 0.7% of the gap it would have to cross to change an ordering, and int8's is 33% — big enough to swap a boundary case occasionally, which is exactly the 1.6% of results it lost. At 1,536 dimensions the ratio is 0.34, and the recall is 0.977. The same ratio predicts the same answer.

What does re-ranking recover?

Re-ranking is the technique people actually deploy: scan everything cheaply, keep a wider candidate list, then re-score just those candidates with the full float32 vectors and take the real top 10.

Recall@10 against the float32 top-10, as a function of how many candidates get re-scored:

For int8 the fix is complete and nearly free. A 10x candidate list costs 100 float32 dot products — 0.15 MB read against the 38.8 MB the int8 scan already touched, a rounding error — and it recovers perfect agreement with the exact answer. int8 plus re-ranking is exact search at a quarter of the storage. That is the recommendation, and it is the only configuration here that gives up nothing.

For binary, re-ranking does not rescue it on this data. You have to re-score a tenth of the entire corpus to reach 0.93, at which point you have read 10,000 float32 vectors (15.4 MB) and the 4.8 MB binary index, and you may as well have scanned float32 directly.

Why is binary so much worse here?

Two reasons, and the first is a genuine property of the format. A binary score is D − 2 × hamming, so at 384 dimensions it can take exactly 385 distinct values, and 100,000 vectors have to be sorted into them. Measured over 20 queries, the top 100 binary candidates spanned only 11.45 distinct scores at 384 dimensions and 19.0 at 1,536. The representation cannot express an ordering finer than that, so the top 10 is largely arbitrary within the top band.

The second reason is the data, and this is where the conservative-floor caveat does real work. Random unit vectors in high dimensions all sit at nearly the same distance from each other: over these 100 queries the mean cosine score was −0.000013, the best neighbour scored 0.2216, and the 10th scored 0.1895 with only 0.001258 separating it from the 11th. Binary quantisation keeps only the sign of each component, and the resolution that survives is nowhere near 0.001258.

Real embeddings have structure that random vectors do not, and published results for binary quantisation with re-ranking on real corpora are far better than 0.062 — this is the format's worst case, not its typical one. But the 385-value ceiling is arithmetic and it does not go away. Take the int8 numbers as a floor you can rely on; take the binary numbers as evidence that binary needs validating on your own data before you trust it, which needs an embedding model and a labelled set, and is a different article from this one.

What should you actually do?

Store float32, and quantise to int8 when the vectors stop fitting comfortably in memory. Below that, the 4x saving buys nothing you can feel: the scan speed is identical and you have added a quantisation step and a scale factor to every write path. Once you are memory-bound — which for the brute-force regime in do you actually need a vector database arrives around a gigabyte or two of vectors — int8 with a 100-candidate float32 re-rank is exact, four times smaller, and has no tuning parameter that can silently degrade.

If you are considering quantisation because retrieval is too slow rather than too large, it is the wrong lever. Halving the number of vectors by chunking documents properly halves the scan time and improves the results, while quantisation halves the storage and leaves the scan where it was. And if the vectors live next to their text in SQLite, as in full-text search that works on a plane, the int8 blob is a drop-in column swap — the scale factor is one extra REAL.

Check it yourself

No network, no API key, no model call, no numpy, no dependencies. Save as quantdemo.mjs and run node quantdemo.mjs — it takes about a second and a half. It builds all four representations from the same 20,000 random unit vectors, then reports storage, scan time, recall@10 against the float32 baseline, and recall after re-ranking 100 candidates.

// quantdemo.mjs -- storage, scan speed and rank agreement for float32 / float16
// / int8 / binary, measured against the float32 baseline on the SAME vectors.
// RANDOM vectors: this measures quantisation error, not retrieval quality.

const DIM = 384, N = 20000, NQ = 20, K = 10, RERANK = 100;

const rnd = (() => { let a = 20260830; return () => { a |= 0; a = (a + 0x6d2b79f5) | 0; let t = Math.imul(a ^ (a >>> 15), 1 | a); t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; return ((t ^ (t >>> 14)) >>> 0) / 4294967296; }; })();
const gauss = () => { let u = 0, v = 0; while (u === 0) u = rnd(); while (v === 0) v = rnd(); return Math.sqrt(-2 * Math.log(u)) * Math.cos(2 * Math.PI * v); };

// --- binary16 codec: Node 23 has no Float16Array (V8 12.9) -------------------
const fb = new Float32Array(1), ub = new Uint32Array(fb.buffer);
function f32tof16(v) {
  fb[0] = v; const x = ub[0], sign = (x >>> 16) & 0x8000;
  const exp = (x >>> 23) & 0xff, mant = x & 0x7fffff;
  if (exp === 0xff) return sign | 0x7c00 | (mant ? 0x200 : 0);
  const e = exp - 112;
  if (e >= 0x1f) return sign | 0x7c00;
  if (e <= 0) {
    if (e < -10) return sign;
    const m = mant | 0x800000, sh = 14 - e;
    let h = m >>> sh; const rem = m & ((1 << sh) - 1), half = 1 << (sh - 1);
    if (rem > half || (rem === half && (h & 1))) h++;
    return sign | h;
  }
  let h = (e << 10) | (mant >>> 13); const rem = mant & 0x1fff;
  if (rem > 0x1000 || (rem === 0x1000 && (h & 1))) h++;
  return sign | h;
}
const LUT = new Float32Array(65536);
for (let h = 0; h < 65536; h++) {
  const s = h & 0x8000 ? -1 : 1, e = (h >>> 10) & 0x1f, m = h & 0x3ff;
  LUT[h] = e === 0 ? s * m * 5.960464477539063e-8
    : e === 31 ? (m ? NaN : s * Infinity)
    : s * (1024 + m) * Math.pow(2, e - 25);
}

// --- corpus -----------------------------------------------------------------
const F32 = new Float32Array(N * DIM);
for (let i = 0; i < N; i++) {
  const o = i * DIM; let s = 0;
  for (let d = 0; d < DIM; d++) { const x = gauss(); F32[o + d] = x; s += x * x; }
  const inv = 1 / Math.sqrt(s);
  for (let d = 0; d < DIM; d++) F32[o + d] *= inv;
}
const F16 = new Uint16Array(N * DIM);
for (let i = 0; i < N * DIM; i++) F16[i] = f32tof16(F32[i]);

const I8 = new Int8Array(N * DIM), SC = new Float32Array(N);
for (let i = 0; i < N; i++) {
  const o = i * DIM; let m = 0;
  for (let d = 0; d < DIM; d++) { const a = Math.abs(F32[o + d]); if (a > m) m = a; }
  SC[i] = m / 127; const inv = 127 / m;
  for (let d = 0; d < DIM; d++) I8[o + d] = Math.round(F32[o + d] * inv);
}
const W = DIM >> 5, BIN = new Uint32Array(N * W);
for (let i = 0; i < N; i++) {
  const o = i * DIM, w = i * W;
  for (let d = 0; d < DIM; d++) if (F32[o + d] > 0) BIN[w + (d >>> 5)] |= 1 << (d & 31);
}

// --- scans ------------------------------------------------------------------
const sc = new Float32Array(N);
const scanF32 = (q) => { for (let i = 0; i < N; i++) { const o = i * DIM; let s = 0; for (let d = 0; d < DIM; d++) s += F32[o + d] * q[d]; sc[i] = s; } };
const scanF16 = (q) => { for (let i = 0; i < N; i++) { const o = i * DIM; let s = 0; for (let d = 0; d < DIM; d++) s += LUT[F16[o + d]] * q[d]; sc[i] = s; } };
const scanI8 = (q, qs) => { for (let i = 0; i < N; i++) { const o = i * DIM; let s = 0; for (let d = 0; d < DIM; d++) s += I8[o + d] * q[d]; sc[i] = s * qs * SC[i]; } };
const pc = (x) => { x = x - ((x >>> 1) & 0x55555555); x = (x & 0x33333333) + ((x >>> 2) & 0x33333333); x = (x + (x >>> 4)) & 0x0f0f0f0f; return Math.imul(x, 0x01010101) >>> 24; };
const scanBin = (q) => { for (let i = 0; i < N; i++) { const w = i * W; let h = 0; for (let k = 0; k < W; k++) h += pc(BIN[w + k] ^ q[k]); sc[i] = DIM - 2 * h; } };

function topK(k) {
  const idx = new Int32Array(k), val = new Float32Array(k);
  let n = 0, th = -Infinity;
  for (let i = 0; i < N; i++) {
    const s = sc[i];
    if (n < k) { let j = n++; while (j > 0 && val[j - 1] < s) { val[j] = val[j - 1]; idx[j] = idx[j - 1]; j--; } val[j] = s; idx[j] = i; if (n === k) th = val[k - 1]; }
    else if (s > th) { let j = k - 1; while (j > 0 && val[j - 1] < s) { val[j] = val[j - 1]; idx[j] = idx[j - 1]; j--; } val[j] = s; idx[j] = i; th = val[k - 1]; }
  }
  return idx;
}

// --- queries ----------------------------------------------------------------
const reps = ['float32', 'float16', 'int8', 'binary'];
const bytes = { float32: F32.byteLength, float16: F16.byteLength, int8: I8.byteLength + SC.byteLength, binary: BIN.byteLength };
const hit = {}, hitRR = {}, times = {};
for (const r of reps) { hit[r] = 0; hitRR[r] = 0; times[r] = []; }

for (let n = 0; n < NQ; n++) {
  const q = new Float32Array(DIM); let s = 0;
  for (let d = 0; d < DIM; d++) { const x = gauss(); q[d] = x; s += x * x; }
  const inv = 1 / Math.sqrt(s); for (let d = 0; d < DIM; d++) q[d] *= inv;
  let m = 0; for (let d = 0; d < DIM; d++) { const a = Math.abs(q[d]); if (a > m) m = a; }
  const qs = m / 127, qi = new Int32Array(DIM);
  for (let d = 0; d < DIM; d++) qi[d] = Math.round(q[d] * 127 / m);
  const qb = new Uint32Array(W);
  for (let d = 0; d < DIM; d++) if (q[d] > 0) qb[d >>> 5] |= 1 << (d & 31);

  scanF32(q);
  const truth = Float32Array.from(sc);
  const gold = new Set(Array.from(topK(K)));

  for (const r of reps) {
    const t = process.hrtime.bigint();
    if (r === 'float32') scanF32(q); else if (r === 'float16') scanF16(q);
    else if (r === 'int8') scanI8(qi, qs); else scanBin(qb);
    const cand = topK(RERANK);
    times[r].push(Number(process.hrtime.bigint() - t) / 1e6);
    for (let j = 0; j < K; j++) if (gold.has(cand[j])) hit[r]++;
    const rr = Array.from(cand).map((i) => [truth[i], i]).sort((a, b) => b[0] - a[0]);
    for (let j = 0; j < K; j++) if (gold.has(rr[j][1])) hitRR[r]++;
  }
}

const med = (a) => { const s = [...a].sort((x, y) => x - y); return s[s.length >> 1]; };
console.log(`node ${process.versions.node}   ${N} random unit vectors x ${DIM} dims   ${NQ} queries`);
console.log('RANDOM vectors: quantisation error only. Says nothing about retrieval quality.\n');
console.log('rep         MB     B/vec   smaller   scan ms   recall@10   +rerank100');
for (const r of reps) {
  console.log(
    r.padEnd(10) +
    (bytes[r] / 1e6).toFixed(1).padStart(6) +
    (bytes[r] / N).toFixed(0).padStart(9) +
    (bytes.float32 / bytes[r]).toFixed(1).padStart(9) + 'x' +
    med(times[r]).toFixed(1).padStart(10) +
    (hit[r] / (K * NQ)).toFixed(3).padStart(12) +
    (hitRR[r] / (K * NQ)).toFixed(3).padStart(13));
}
console.log('\nfloat32 must read 1.000/1.000 -- that row is the control.');

On the M3 laptop above that prints:

node 23.5.0   20000 random unit vectors x 384 dims   20 queries
RANDOM vectors: quantisation error only. Says nothing about retrieval quality.

rep         MB     B/vec   smaller   scan ms   recall@10   +rerank100
float32     30.7     1536      1.0x       8.7       1.000        1.000
float16     15.4      768      2.0x      10.8       1.000        1.000
int8         7.8      388      4.0x       9.2       0.990        1.000
binary       1.0       48     32.0x       0.4       0.060        0.320

float32 must read 1.000/1.000 -- that row is the control.

Check the control row first: if float32 does not read 1.000, the harness is broken and nothing else in the output means anything. Then change RERANK to 10 and watch int8 drop to 0.990 — the whole argument for two-stage retrieval, in one constant.