Google has begun developing artificial intelligence (AI) inference chips with semiconductor design firm Marvell Technology. The move marks Google's swift counterattack as Nvidia seeks to dominate the inference chip market with its language processing unit (LPU), following its conquest of the general-purpose graphics processing unit (GPU) market. Marvell had been a partner for Nvidia's inference weapon "Grok," meaning Google has effectively pulled in its rival's ally.
The Information reported Wednesday, citing sources, that Google is in discussions with Marvell to develop two new AI chips.
One of the two chips is a memory processing unit (MPU) designed to work in tandem with Google's existing in-house AI chip, the tensor processing unit (TPU). The other is a new inference-focused TPU.
Marvell's MPU does not merely store data but directly handles part of the computation. Conventional designs consume power and create delays as data moves between the TPU and memory, but the Marvell MPU shortens that path. Sources said Google and Marvell plan to finalize the design by next year before entering trial production, with plans to manufacture approximately 2 million MPUs.
The other chip, the new TPU, is an AI chip succeeding Google's flagship seventh-generation TPU "Ironwood." While the TPU started as a chip dedicated purely to inference, Ironwood evolved into a general-purpose high-performance chip handling both training and inference. The new TPU is an advanced inference-specialized chip that is cheaper and more power-efficient. Google began selling its internally used TPUs to competitors last year as the AI market shifted toward inference.
Google has threatened Nvidia's GPUs since 2018 by releasing TPUs for data centers. GPUs are strong in AI training but weak in inference, where speed is critical. Google targeted this gap, signing a multibillion-dollar TPU leasing agreement with Meta and forming a rivalry with Nvidia.
However, tensions rose at Google when Nvidia unveiled its LPU last month. Nvidia revealed its ambitions by indirectly acquiring inference chip developer Grok late last year and then launching the "Grok 3 LPU." The Information, citing a Google employee, reported that "Google has accelerated development of its new inference chip since Nvidia's LPU reveal," adding that "Marvell was Grok's partner during the development of the first-generation LPU, so it also has experience designing inference chips."
