It has been a busy season for Digital Infrastructure conferences and particularly those focussed on or featuring data centers. There are literally three topics dominating every conference and forum lately. Those are Artificial Intelligence (AI), specifically Generative AI, Sustainability, and the current Geopolitical landscape. Why are these topics coming up over and over? Because all three threaten to have significant impact on the Digital Infrastructure landscape as we know it.
Generative AI
AI has been around as a topic for many years now, but it made little impact on our daily lives. Then in November of 2022, everything changed with the emergence of Large Language Models (LLM) and this new concept of Generative AI – the ability for an LLM to create new media – text, graphics, speech, and even videos. The LLMs do this by ingesting massive amounts of previous data (generally called training) and then referencing that data in order to generate new content. These models are split into two key functions: the previously mentioned training and inference – the act of generating new content. Both functions run on a specific type of computer chip called a Graphics Processing Unit (GPU). [Note that Google and has their own version called a Tensor Processing Unit (TPU)]. These GPUs are much more power hungry than traditional CPUs and therefore, generate much more heat in a smaller space than traditional processors/servers. While both functions run on the same hardware, they each impact Digital Infrastructure, particularly Data Centers, in different ways.
Inference is the act of GAI looking at inputs and predicting what the user wants. Inference, due to the nature of it interacting real-time to those inputs, needs to be housed closer to the service for which it is making these inferences. Think of Google predicting what you will type next when searching for something. These will naturally be hosted in or near the same facilities in which the underlying services are hosted. Examples would be Microsoft’s Bing and ChatGPT-4 integration or Google’s Bard. These inference installations are not particularly large and as a whole, will be highly distributed in data centers globally.
Training is the act of feeding huge amounts of data to a LLM, having it interpret it, and then correcting any deviations, effectively teaching it how to generate new media. As such, these LLM training models are very large and require enormous amounts of power in relatively smaller spaces, creating a problem rejecting all the heat they produce. This fact alone will drive changes in Data Center design and bring forward technologies including liquid-based cooling. These facts will likely cause the owners of these training models to congregate in fewer, larger installations and they will likely gravitate to geographies where there is cheaper power and more efficient forms of cooling available. This will drive the creation of an entirely new data center – one that is AI ready and may even open up new geographies to large data center installations, for example in Asia such as Malaysia and Perth in Western Australia, where power is amongst the cheapest in the world.
Sustainability
The ever-increasing demand for power in a smaller footprint, known as power density, will bring sustainability even more to the forefront.  Of huge importance will be the availability and price of renewable energy and an increased need for efficiency, specifically in cooling the servers and in not evaporating huge amounts of water to do so.  To date, sustainability, particularly as defined above, have been a ‘nice to have’, but not a ‘must have’ for the data center industry as a whole. It is estimated that the global data center industry currently consumes approximately 1% of the global energy produced each year. Some estimate that it is actually much higher. With the increased demand of GAI and other factors, like countries shutting down coal and gas-fired power plants, the need for sustainable energy coupled with more energy efficiency will become paramount. If the industry is not careful, it may be seen as energy thieves, taking power from other sectors and the general populous. Furthermore, the transmission and distribution systems in most countries are ill prepared for the increased demand of power amidst the retirement of fossil fuel power generation. Governments must act quickly to not only enable more renewable energy sources to come online, but improve the energy transmission and distribution systems, as well as allow for easy connection to the power grid and/or direct connection of the offtake (e.g. data centers) to those renewable energy sources without burdensome regulation.
Geopolitical Landscape
The geopolitical landscape has been dominated lately by the US-China tech break. It has impacted not only equipment manufactures (e.g. Huawei), but chip manufacturers (e.g. SMIC) and even sub-sea cables that are the global connectivity that makes the global Internet work in the first place. Adding further tensions and uncertainty are China’s actions in the South China Sea and their posture toward Taiwan. The vast majority of these increasingly important GPUs are manufactured in Taiwan and the remainder are manufactured in Hong Kong. There is huge risk in China dominating the technology that underpins Generative AI. As for subsea cables, many important cables pass through the South China Sea, exposing them to potential actions by a hostile force.
Conclusion
The Digital Infrastructure space, including the Data Center industry is at an inflection point, which is being influenced by three major factors. The rise of Generative AI will drive a fundamental redesign of part of the market as well as the need for more efficiency and sustainable energy. The geopolitical landscape is highly diverse and impacted by the actions of individual countries, regional blocs, and regulations, which vary by country. We need to navigate these choppy waters carefully, but at the end of the day, the Digital Infrastructure industry is going to continue to grow at a record pace and much new innovation is on the horizon.
