Google’s TurboQuant has officially ended the era of “solving AI with more RAM.” Chip companies are going down, perhaps the trend is now over. This new research breakthrough from Google Research solves the biggest problem in large language models: the memory-hungry KV cache.
By using two key techniques; PolarQuant and QJL. TurboQuant compresses the memory needed to run AI by six times. More importantly, it does this with zero loss in performance. On high-end hardware like the Nvidia H100, it can even boost processing speeds by up to eight times. Amazing right?
The semiconductor industry felt the impact immediately. Major memory chipmakers like SK Hynix, Samsung, and Micron saw their stock prices tumble by 4 – 7%. Investors are spooked because if software becomes this much more efficient, the desperate need for massive physical memory upgrades might vanish. Is this the end of the chip bullmarket?