Keeping Your Cool in a World of Hot Compute: AI, Bitcoin & Liquid Cooling

Hello, everyone, and welcome to our panel. My name is Jaran. I work for E2C Partners, a data center consulting company, and I came all the way from Cyprus to moderate this panel. First, I’ll let the panelists introduce themselves. Matthew, please start.

​

Hi, I’m Matthew Carson, chief business officer of Hash House. We design, engineer, and manufacture immersion and hydro cooling infrastructure for mining.

​

Hi everyone. Very nice to meet you here. At lunchtime, you are here, so you must be hungry for the future and hungry for knowledge. I’m very honored. I’m Suelyn from Changzhou Bingrui Heat Transfer Technology.

​

Hello, everybody. My name is Jeremy Singer with ExxonMobil. I’m here representing our immersion fluid line. My history involves many years in HPC and AI compute, and also low carbon data centers.

​

Matthew, you are a system integrator of data center cooling solutions. Can you give everyone an overview of data center cooling, why it is important, and what the challenges are?

​

These days there are really three main technologies. There is air cooling, which I think we are all very familiar with. There is hydro cooling, which uses water direct to chip with large coolant distribution units and large dry coolers to cool down the servers and equipment. Then there is immersion cooling, which is the physical dunking of servers in fluid for the purpose of cooling them down.

​

Is cooling fully solved, or are there still a lot of inefficiencies that could be solved over time?

​

I think there are still a lot of inefficiencies being worked on. In mining, especially, a huge one is heat reuse, particularly in Europe with district heating, power plants, and the heating and drying of wood or vegetable products. There is a lot of low-hanging fruit for better use of the byproduct of these cooling systems, which is heat. But in terms of the cooling systems themselves, they have gotten very advanced very quickly, depending on your needs and uses.

​

Adding to that, technology matures at a very rapid pace. We are all familiar with Moore’s Law and how chips improve their speed on a year-over-year cycle. Cooling infrastructure is very similar. It is a physics game where efficiency matters. Every dollar and every joule of energy being used is most advantageously applied if you are as efficient as possible. There are a lot of technology opportunities to continue to mature there, and eventually there will be a roofline where these challenges face us head on. I think there is continued work to do in the cooling space.

​

Have miners been the ones who really made all the innovation in the data center cooling industry over the past years, or who has been driving that innovation? Is it miners or traditional data centers?

​

I think it is still split pretty heavily in favor of traditional data centers. They have much larger R&D budgets and much larger capital expenditure budgets. But on the lower, leaner end of things is probably where innovation happens in the Bitcoin mining space. We are all technically competing against each other globally, so we need to get the absolute best efficiency and the absolute best pricing for our infrastructure, down to a much more detailed level than at least traditionally existed in the data center space. Now that AI and HPC have exploded, I think they are starting to see the same constraints around capital efficiency, and they are starting to learn some of the lessons we have learned over the last couple of years.

​

Suelyn, you do heat exchangers, which are a very specific part of the cooling infrastructure. Can you talk about the different components in the cooling system and the most important parts that data centers may overlook?

​

Definitely. I’m pleased to introduce aluminum plate heat exchangers. We have worked for ten years in the mining sector and related applications. In the past, the older traditional structures used steel or copper. The heat transfer efficiency was not so high. Using aluminum with a more complex design and high heat transfer efficiency can reduce a great deal of weight. The heat transfer efficiency is very high, and it already has big applications in different sectors. It is mature technology in this sector.

​

I think it is revolutionary for this kind of application in mining and in the AI sector, especially for GPUs. CPUs have already started to use this kind of product, and mining as well. Most people, no matter whether they choose the machine-type mining method or the cooling method, are starting to use aluminum. I think this is very good for the future.

​

Going back to your prior question for a second, around the maturity of the various cooling technologies, I think you can always trust a Bitcoin miner to do something a little radical in pursuit of efficiency and optimization. For that reason, I give a lot of credit to early Bitcoin miners trying immersion and trying things in weird places. Typical data centers at scale are going to be in high-capital, intense areas with raised floors, lots of good air conditioning, and perfect environmental controls. That is good for certain technology, but you can trust the Bitcoin miner to innovate and iterate to find new technologies. Immersion is a good example. Miners brought that to scale earlier, and it is starting to take off a little bit in the data center world, depending on the technology.

​

When we talk about different cooling methodologies, it is air, immersion, and hydro. What are the main differences, trade-offs, and advantages, and in what applications do they work better than others?

​

For air-cooled mining, the PUE can be around 1.5 to 1.8. Electricity price is the main cost of mining operations. If you choose hydro cooling systems, this can help reduce costs and create a great deal of savings. If you try to reduce your PUE, it may be more important than only choosing based on electricity price. Use advanced technology to save cost and increase profit.

​

Some of the other differences between hydro, air, and immersion are that air, in general, is far inferior to hydro and immersion in terms of power utilization efficiency. In general, hydro and immersion can be 30% to 45% more efficient than air. When it comes to hydro versus immersion, it is really about the customer. Both have pros and cons, similar efficiency, and similar capital requirements, but there are operational differences between hydro and immersion, and it can be highly site and customer dependent.

​

In the eternal debate between hydro and immersion, especially these days as the technologies in the mining space have become fairly mature, it really comes down to what the customer is looking for. In a very cold climate, I might recommend immersion, as the fluid generally has a lower freezing temperature, fewer issues with expansion and contraction in colder temperatures, and less chance of a small leak on the floor of a container causing ice and someone slipping and falling. If we were looking at a hot desert environment, I might recommend hydro, as most immersion designs are open-tank designs, and the chance of foreign debris entering the system, like sand and dust, is quite high. In those circumstances, I might recommend hydro.

​

So overall it is a trade-off, and it depends on the environment where you want to deploy the infrastructure. What are the CapEx differences between these three cooling systems?

​

From our experience, air is generally the cheapest option. It has the least number of subcomponents and subassemblies. But for mining, the price difference between immersion and hydro is actually very close. For most customers, the quoted system price ends up being pretty much within a rounding error. I’m not sure what the experience is on the HPC side, but I would imagine it is quite a bit different.

​

Depending on the scale of the HPC system, how much infrastructure you need, what your floor and white space look like, and the build-out costs, that can impact whether you go with hydro or immersion. Your operational support staff can also be impacted by this. With hydro, you are talking about closed-loop systems, cooling distribution units, and a simple fluid that circulates like a circulatory system. With immersion, there are more varieties of tank types and oil choices, and maybe more operational support considerations in terms of training staff and places to drain equipment for support and maintenance.

​

Speaking for immersion, it is a little simpler. You literally have a tank full of dielectric oil that does not conduct electricity. You put the machines in it. If the tank is designed appropriately, and the designs have matured dramatically, the systems stay in a comfortable temperature range, reliability is increased, and you do not have to touch them as much. In the hydro world, there can be a lot of interconnection points and opportunities to drip. Hydro fluid does conduct electricity, and it can be corrosive depending on the hydro fluid you are using. These are some of the operational considerations to think about when deploying at scale.

​

Hydro and immersion are about the same cost in terms of CapEx. You mentioned a lot of challenges and nuances in running hardware in immersion operations, depending on geography, climate, dust, and other factors. Can you give a rough number on the OpEx difference? Maybe one requires fewer employees. Are there differences in OpEx?

​

For mining at scale, a well-done immersion system, whether containerized or building skid-mounted, should generally require fewer people to keep running and is more operationally forgiving. With hydro, especially larger systems with high pressure and high flow rates, the water has to move very quickly in order to remove and move that much heat. The whole system tolerance is much tighter than immersion.

​

With immersion, it is easier to leave a couple of miners out if they are being repaired or need to be fixed. In hydro, you have to be very careful about overall system flow and pressure. If you remove too many miners in one specific rack, you may have to power down more of the entire system to make sure your cooling loop stays within parameters.

​

So you would say immersion is a bit more modular and easier to operate?

​

Overall, yes.

​

Suelyn, with heat exchangers, are there any differences between immersion and hydro?

​

Aluminum heat exchangers can be used in both immersion and hydro. They are very easy to maintain. You do not need to spend a lot of time thinking about blockage and leakage, because every unit is modular and liquid-tested before shipping. The connections are very easy, and the system can be put into operation very fast.

​

In both immersion and hydro, they are friendly to aluminum and operate in a closed circuit without evaporation of the liquid. At the beginning, the investment cost may be higher than air cooling, but over the long run, hydro and immersion are much more effective, easier to maintain, and have a long operating life.

​

The liquid is extremely important in immersion cooling. Jeremy, can you talk about that liquid and what ExxonMobil does with it?

​

Absolutely. At a high level, what we are talking about is an oil-like substance that does not conduct electricity. You put the mining computer inside it, literally the whole system inside a tank. The tank flows the fluid through a heat exchanger and removes the heat externally.

​

The cooling fluids do not conduct electricity. They may be synthetic in nature, created with chemicals, petrochemicals, or other chemicals designed to remove heat, avoid igniting, and avoid conducting electricity. Across the landscape of fluids, they all have similar properties. Some may have different viscosities, so they flow easier or harder depending on temperature. Some may have different flashpoints. All of them should have flashpoints that are higher than what you need for compute, meaning the compute will turn off way before the fluid will catch fire. They should be safe to operate and nontoxic.

​

The ExxonMobil fluid hits these attributes. It is a high-quality fluid that will last a long time, remove heat without degrading, and flow very well from a viscosity perspective.

​

Let’s move over to Bitcoin mining versus AI compute and other forms of data centers. What are the differences in cooling a Bitcoin mining data center, an AI data center, and a traditional data center?

​

I’ll start. I have mined personally, though not at the scale I aspire to, and I have built, designed, run, and supported multiple top-20 supercomputers in the world. Supercomputers are very finicky beasts. They need to stay cool. The workloads we practice at ExxonMobil are highly intensive, all-to-all communications that require all processors and all network devices to talk to each other with full fidelity. You take a massive math problem, split it into millions of parts, execute those parts in parallel, sync at the appropriate times, and then create a work product. In this case, we are talking about subsurface imaging, or making pictures underneath the earth of what oil reservoirs may look like so you can optimize those.

​

To do that effectively, a supercomputer needs to be fully online and fully communicative with itself. It needs to talk to each other with high integrity. Bitcoin mining is not like that. Each miner operates independently. Yes, you are mining together as a pool, but it is really a function of how much power you are consuming and how much hash you are creating. Whether a certain miner is finicky or not communicating does not matter as much as it would in the supercomputer or AI training world, where a node that goes out during a training run can disrupt the entire run and potentially invalidate it.

​

I would treat Bitcoin mining more like an inference workload, more of a question-and-answer workload where, if I ask where the bathroom is, it is okay if it takes an extra second to answer, or if it gets something small wrong. Those are little problems I am less concerned about compared with when I want it to be absolutely right in a training load.

​

Cooling matters here. Cooling resiliency and cooling quality matter. If your cooling resiliency is low and your nodes are falling out erratically, you are not going to get much work done in the AI world. The integrity required in AI from a cooling perspective is much higher. But the Bitcoin world eats a lot of power. In fact, the more power you use, the more money you make, so you want to stay cool for long-term resilience.

​

Building off that, it really comes down to resiliency and the corresponding redundancy requirements. From a cooling system or cooling fluid standpoint, your fluid can contain a certain number of joules of heat that can be rejected. That is physics. We know your dry coolers will have a certain amount of heat rejection capability based on certain ambient temperatures and fans installed. Those factors do not really change.

​

What changes is that my hydro-cooled Bitcoin miner might be able to accept anywhere from minus 20 C to 80 C fluid intake temperatures, while most HPC or AI servers would likely have major problems if you tried to feed them 50 C fluid intake, regardless of what fluid you choose. You have cooling redundancy and cooling delta T differences between HPC, AI, and Bitcoin. Then you talk about redundant networking, redundant security systems, and redundant power systems, all of which are much more important when you need to keep your whole load live and happy.

​

With Bitcoin mining, if I have a dry cooler fail on one of my containers, that does not take down my whole site-based load. I just lose the proportion of my income that particular container represents in the overall project.

​

We have already touched on the nuances in cooling depending on geography, climate, and other factors. How do different geographies impact power usage effectiveness?

​

In places like Canada or areas near the North Pole or South Pole, the temperature is very different, and there is usually very low consumption for coolers. For hot areas, especially places with water shortages, like the Middle East, immersion may be more suitable. From a corrosion perspective, if the product is very close to the sea, it needs anti-corrosion treatment.

​

Usually when we design heat exchangers, we make the design based on the installation city. We first ask where the mining farm will be located. If it is in a shore area, on a plateau, or in a mountain area, we choose different designs for the situation. We make sure the heat exchanger and the system will perform well and meet the most demanding situation.

​

Building on what Suelyn said, the location of the client site is probably the most important piece of information they can give us. Detailed weather information and altitude information all go into what dry cooler we select, what fluid we select, and what combination of systems we select. If we are talking about building a mining farm in the UAE in the middle of the desert, where we have 50 C ambient temperatures, then besides the fluid itself, the whole system is going to be designed completely differently than one I might install in Alaska or Alberta. They are similar only in function, not much in form.

​

What about heat reuse? Very quickly, do you think heat reuse will be a big thing for data centers in the future all over the world, or is it more of a niche opportunity right now?

​

In terms of absolute scale, it is probably a niche opportunity when you look at the total amount of power consumption for compute versus how reasonable it is to replace some hot water systems. But in terms of energy requirements, heating water is one of the largest uses of energy across the entire world. Both Bitcoin mining and HPC or AI, when paired with proper systems, can become step-in replacements for water heating systems. I see a very large future in heat reuse and heat recapture systems in our industry.

​

I agree with that. Heat reuse is largely a CapEx and energy optimization problem. Energy, from my perspective, is a very precious resource, and we should seek to optimize it wherever possible. It is kind of a niche right now, but with CapEx and creative solutions, there is future potential. I am looking forward to getting all the energy out of every molecule possible.

​

Thank you for coming, everyone.