Nvidia is switching to liquid cooling to reduce the power consumption of large technologies

Nvidia has announced a new plan to reduce power consumption in data centers that process massive amounts of data or train AI models: liquid-cooled graphics cards. The company announced at Computex that it is introducing a liquid-cooled version of its A100 calculation card, and says it uses 30 percent less energy than the air-cooled version. Nvidia also promises that it’s not just a one-off, but already has more liquid-cooled server cards on its roadmap, as well as tips on how to transfer this technology to other applications, such as automotive systems that need to stay cool in confined spaces. Of course, Tesla’s recent withdrawal from chip overheating shows how difficult this can be, even with liquid cooling.

According to Nvidia, reducing the energy needed to perform complex calculations can have a big impact – the company says data centers use more than one percent of the world’s electricity, of which 40 percent comes from cooling. Reducing that by almost a third would be a big deal, although it’s worth noting that graphics cards are only one part of the equation; Processors, storage, and networking hardware also draw energy and also need cooling. Nvidia says that with liquid cooling, GPU-accelerated systems would be much more efficient than CPU-only servers for AI and other high-performance tasks.

Nvidia roadmap for liquid cooled devices and cards.
Photo: Nvidia

It’s not for nothing that liquid cooling is popular in high-performance applications, from supercomputers to custom gaming PCs and even a few phones: According to Asetek, a major manufacturer of water cooling systems, liquids absorb heat better than air. And once you’ve got warm fluid, it’s relatively easy to move it elsewhere to let it cool down, compared to trying to cool the air throughout the building or increase the airflow to specific elements on the card that dissipate all the heat.

In addition to being energy-efficient, liquid-cooled cards have another advantage over their air-cooled counterparts – they take up much less space, meaning you can fit more of them in the same space.

Nvidia’s emphasis on reducing power consumption through liquid cooling comes at a time when many companies are considering the amount of power their servers use. While data centers are far from the only source of carbon emissions and pollution for big technologies, they are part of a puzzle that cannot be ignored, and critics have noted that offsetting energy use with credits isn’t as effective as reducing overall consumption. Companies like Microsoft have experimented with completely submerging their servers in liquid, or even placing entire data centers in the ocean to use less energy and water.

Of course, these solutions are rather exotic – while the type of liquid cooling offered by Nvidia is not necessarily the norm for data centers, it’s not as obvious as placing servers in the ocean (though so far Microsoft has experimented with it shockingly successful). Nvidia clearly advertises its liquid-cooled GPUs as being designed for “mainstream” servers and not as a state-of-the-art solution.

This raises the question, could we possibly see Nvidia even try to apply liquid cooling? still mainstream by embedding liquid cooling into the reference designs of your gaming-focused cards. The company doesn’t mention any plans to do so, just says it plans to “support liquid cooling in our high-performance data center GPUs” in the “foreseeable future.”

However, server technology continues to flow into home computers, and game cards shipped straight from the factory with an all-in-one liquid cooling system are not entirely unheard of – AMD had several reference designs that included the liquid-cooling loop, and third-party previously sold liquid-cooled Nvidia cards. As Nvidia cards continue to draw more and more power (the standard 3090 Ti can draw up to 450 watts), I wouldn’t be surprised if Nvidia announced the RTX 5000 series card that comes with a liquid cooler at the factory.

As for Nvida cards targeting data centers, the company says companies such as ASRock, Asus and Supermicro will be introducing liquid-cooled cards to their servers “later this year”, and PCIe A100 cards with a socket will arrive in the third quarter of this year. A liquid-cooled PCIe version of the recently announced H100 card (which is the next-generation version of the A100) is expected to arrive “in early 2023”.

Leave a Reply