Immersion cooling – the next big step in data center cooling?

Data centers will face some issues over the next few years. They consume huge amounts of energy, a large proportion of which is used for cooling. In some European countries, there is legislation requiring them to reuse at least parts of the heat for data centers by 2027, and there are minimum efficiency requirements for new and, going forward, existing data centers. How can this be achieved? Immersion cooling can be a solution, although it would require enormous changes in the data centers and substantial investments.

Why immersion cooling?

It is well known that air is not very suitable for cooling. However, it is used anyway, primarily because this is straightforward, and air is not electrically conductive. That is why air cooling systems are still the method of choice in data centers. However, with increasing calls for sustainability, particularly for using waste heat in combination with higher packing densities, cooling must be reconsidered.

Waste heat must have a specific temperature to be used efficiently. Today's data centers usually operate at 27°C. Using waste heat for district or local heating would require temperatures of 75°C and more. Although heat pumps can achieve these temperatures, they are most efficient when the temperature difference is as small as possible.

Efficiency requirements such as PUE (total energy consumption of the data center/energy consumption of IT) are mainly affected by the efficiency of cooling and energy supply, with cooling having the biggest impact. That means optimization is crucial here.

Liquids have the advantage of being able to transport heat much more efficiently. The temperature can be much higher, as the buffering capabilities are much higher than with air, and the quantities to be moved can be much smaller, significantly increasing the pumps' efficiency.

Liquid cooling is not new in IT. Even the first mainframes were cooled with water.

Immersion cooling is anything but mainstream, though. Microsoft and Dug McCloud publicly mentioned the first implementations. Nowadays, they are discussed more frequently, and the first solutions are available in the market, such as Iceotope, Schneider Electric, and Green Revolution Cooling. Technically speaking, there are two basic architectures: open bath immersion cooling and enclosed chassis immersion cooling.

• Open bath immersion cooling is a cooling technology for data centers in which the IT equipment is completely immersed in a dielectric liquid. These baths allow the movement of the cooling liquid through the servers immersed in them. The main advantages of this approach are that practically any type of server can be used, complete racks can be immersed in the bath, and the waste heat can be processed centrally. A downside is that the weight of these filled basins is significantly greater than normal rack specifications allow, so many data center floors are not designed for this.
• In sealed server immersion cooling, the servers are housed in liquid-tight enclosures. The dielectric coolant circulates inside the servers or is pumped through them to absorb the heat from the components. The heated liquid is directed to a heat exchanger in the rack, where it is dissipated. The main advantage of this approach is that the servers are mounted in closed containers that can be replaced in the rack without the liquid being accessible. A disadvantage is that not all hardware can be used, as the manufacturer determines the hardware specifications of the sealed servers.

Regulations for data centers

There are certain limitations to the growth of data centers and their energy consumption:

• Power supply limitations – In some regions, the local energy supply does not allow for new data center loads.
• Shortage of suitable land for building – The requirements for data center locations are high. Floods, earthquakes, and hurricanes must be ruled out. There has to be a secure water supply and, even more critical, electricity. Last but not least, connectivity in the form of Internet, carrier Ethernet, and the like has to be available with maximum bandwidths.
• Community resistance – More and more societies focus on sustainability and the environment. Data center operators must be good neighbors, explaining why their campuses will benefit the community and not cause noise-related issues, be a monstrosity, or drain precious resources.

In addition, there are new requirements such as using waste heat (which only makes sense if the data center is near a village or city), using only renewable energy sources, meeting certain efficiency classes in terms of PUE, and reporting obligations related to ESG. For new data centers, the EU has established the norm EN 50600, which comprehensively defines how a sustainable data center must be built, from the building to energy to reporting.

All this leads to new cooling requirements for more efficiency and higher packing densities.

Are there alternatives to immersion cooling?

The biggest energy guzzlers in the cooling sector are fans and air conditioning technology. In some regions, air conditioning technology can be dispensed with to some extent, and free cooling can be used, which will reduce energy consumption significantly. However, this is not an efficient solution to the challenge of using waste heat.

An alternative to immersion cooling is liquid cooling using heat exchangers at the CPU/GPU level. The advantage is that the weight of the liquid does not play a major role. However, there are major disadvantages: The liquid temperatures are usually lower than those of immersion cooling because the buffering capacity is relatively small. The liquid has to be pumped, which is energy-intensive for large installations, and the size of the heat exchangers prevents higher packing densities.