Optimising Data Centers for AI: Power, Cooling, and Design Explained

The high power demand of AI workloads is mounting pressure on data centre designs, with the technology evolving faster than we can build. While many conversations to tackle emerging challenges revolve around cooling the server load, power and electrical design also play pivotal roles in hosting AI loads. The impact of AI is already being observed today through the massive success of the GPU manufacturer Nvidia.

Nvidia has captured roughly 80% of the AI chip market. In 2023, Nvidia reported $29.12 billion in data centre revenue, showcasing the booming demand for AI-centric hardware. Companies like AWS, ARM, AMD, and Intel are also revolutionising the AI chip market, driving competition and rapid advancements.

To understand this more, we welcome Peter Gross, Managing Partner of PMG Associates Consulting and Advisory as the guest of i3TV with Ed Ansett – Founder of i3 Solutions Group, Luke Neville – Managing Director of i3 Solutions Group, Kerr Johnstone – Director at i3 Solution Group and Damien Wells - Founder of Spa Communications discussing the challenges with AI workloads and how to face them. We focus on the challenges of cooling, power requirements and design.

The load profiles of large language models and generative AI differ drastically from those of 'traditional' data centres. GPU power demand can spike, having knock-on effects on power distribution and utility infrastructure. To minimise latency in these models, GPUs need to be tightly packed increasing demand for high-density racks.

AI workloads require significant power and cooling solutions, with AI models running across thousands of processing cores and servers drawing up to 100 kW per rack. There are immense power and cooling implications, requiring robust solutions to manage the increased energy and heat levels.

This video outlines the mechanical and electrical engineering challenges that will emerge over the next few years. We also discuss the strategic importance of location and sustainability. We will learn how hyperscalers like AWS, Microsoft, Google and Oracle will adapt to meet this novel demand.

0:00 - Intro
0:35 - GPU Power Behaviour
4:46 - Rack Density
6:22 - Cooling
11:20 - Power and Location
15:47 - Training vs Inference Data Centres
18:38 - Design
22:26 - The Future
24:45 - Geopolitics
26:00 - Manufacturing Changes


Read more: https://datacentrereview.com/2024/03/...

Get to know us:

Website – https://i3.solutions/
Blog – https://i3.solutions/blogs/
GHG Abatement Whitepaper Series - https://i3.solutions/ghg-abatement/
Leadership team – https://i3.solutions/leadership/
LinkedIn -   / i3solutionsgroup  


#AI #datacentres #hyperscalers #google #microsoft #aws #nvidia #gpu #aichips #liquidcooling #datacentrecooling #immersioncooling #powersolutions #aitech #generativeai #llm #largelangaugemodel #engineering #electricalengineering #mechanicalengineering #it #airesearch #digitalinfrastructure #itinfrastructure #datacentretrends #directtochip #energy #power #cooling