Menu

Post image 1
Post image 2
Post image 3
Post image 4
Post image 5
Post image 6
Post image 7
Post image 8
Post image 9
Post image 10
Post image 11
1 / 11
0

How a 2021 Quantization Algorithm Quietly Outperforms Its 2026 Successor | Towards Data Science

Towards Data Science·Amit Portnoy·about 1 month ago
#saNCaN81
Reading 0:00
15s threshold

[3] , an online vector quantization method, drew wide public attention at ICLR 2026. For me, it looked very familiar: it overlaps heavily with EDEN , a quantization method first introduced as the 1-bit method DRIVE at NeurIPS 2021 [1] and generalized to arbitrary bit-widths at ICML 2022 [2] . Co-authored by myself, with Ran Ben-Basat, Yaniv Ben-Itzhak, Gal Mendelson, Michael Mitzenmacher, and Shay Vargaftik. The TurboQuant paper presents two variants: TurboQuant-mse and TurboQuant-prod . In a detailed new comparison [5] we show that TurboQuant-mse is a degenerate case of EDEN, and that the EDEN variants consistently outperform their counterparts. How EDEN quantizes a vector Suppose you need to compress a d d -dimensional vector x x (a gradient update, an embedding, a KV-cache entry) down to a few bits per coordinate. EDEN proceeds in four steps: Random rotation — Multiply by a random orthogonal matrix Π \Pi .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More