Water-Cooling the Modern Data Centre

We have a data centre that recently became the first in South East Asia (or perhaps in Asia too) to operate a certain type of liquid cooled rack. Liquid cooling technology is actually not new; It was quite common maybe 2 decades ago. The large mainframe computers of that era produced so much heat that they needed to be cooled by chilled water or refrigerant. Liquid cooling has become less common since then. Nowadays, the thought of water in a data centre is unimaginable to many data centre operators. But today, the advent of very high density computing is beginning to re-introduce liquid cooling into data centres.

After the mainframe era, the trend was for computers to become smaller, consume less power, and produce less heat. Today, computers have indeed become smaller very rapidly, but the trend for power consumption and heat dissipation doesn’t seem to have followed. Powerful computers have shrunk into 1U form factor (just 1.75″ height), and then further into “blades” that are slotted into chassis. We have gone from single-core CPUs to quad-core CPUs. Today, we have actual production server racks with 15KW of load, and mind you that’s actual load (as opposed to rated specifications). That is equivalent to like 15 ovens in a 1000mm x 600mm floor space!

Heating has been a perennial problem in our data centres. At one of our old sites, our greatly oversized (so we were told) air conditioning systems just could not maintain an ambient room temperature of 28 degrees celcius, let alone the “ideal” 22 degrees celcius we would love to achieve. When we finally moved out of that data centre, we observed that once all the servers were gone, the room temperature dropped to as low as 10 degrees celcius! We did indeed have damned powerful cooling! But it was no match for modern day high density computing.

One of the most unusual sights in our latest data centre is the network of huge chilled water pipes running under the raised floor system. While others are keeping water out of their data centre, we have ours filled with water pipes. It is not so bad if you use refrigerant, because if the pipe springs a leak, the refrigerant simply evaporates and you are left with little mess. With water, of course, a leak could mean a flooded data centre. Water don’t mix well with electricity, the reason why most sites are very eager to keep water out of the IT server area.

Why chilled water? It turns out that chilled water is a lot more effective and efficient at carrying away heat than refrigerant. Most water-cooled solutions at present, however, are focused on bringing chilled water to a cooling coil nearer to the heat source, rather than using the chilled water to directly cool the heat source. The problem with the latter is that significant engineering changes are required for severs, storage and other IT equipment to accommodate water circuits.

Now if you are thinking about those water heat sinks and radiators you have come across in Sim Lim Square… this is not quite the same thing. Those Sim Lim Square PCs do not need water. The fanciful cooling is just that: fanciful. They typically don’t even need air-conditioning. 🙂