Artificial intelligence data centers are on the verge of reaching their limits.
To meet ballooning demand, chipmakers like Nvidia Corp. are churning out ever more powerful chips, requiring a new generation of data centers that will draw many times more power than their predecessors.
Rampant power consumption, fueled by “AI factories” that gobble up enough power to keep the lights on in millions of homes, threatens to put more pressure on electricity prices in the U.S., expand AI’s carbon footprint — and potentially slow down the AI boom.
Political backlash against data centers is already creating friction, and industry leaders warn of another more basic constraint: the limits of power generation.
“Very soon, maybe even later this year, we’ll be producing more chips than we can turn on,” Tesla Inc. and SpaceX CEO Elon Musk said earlier this year.
Demand, however, continues to surge. Trillions of dollars are expected to flow into the AI build-out, raising the prospect that energy shortages could become one of the biggest brakes on AI’s growth.
This crunch is forcing a reckoning among key AI players — hyperscalers, data center operators, chip makers and power equipment producers. As they scale up, they are having to reimagine how data centers are designed, built and powered.
Traditional data centers that support services like cloud storage, e-commerce and web hosting use chips known as central processing units, or CPUs. These tasks are typically far less energy-intensive than AI processing.
A standard server rack in this kind of data center might require between 25 and 40 kilowatts, or enough to power around 20 air conditioners.
But AI data centers run on densely packed, more advanced graphic processing units, or GPUs. As AI models get faster and more powerful, racks are packing far more chips in the same space.
“Increasingly, the rule of the game now in AI is that the more you can pack performance in a chip, the densities will keep getting higher and higher,” said Sachin Jain, chief operating officer at CoreWeave, Inc., a cloud provider.
Right now, around 30% of the power flowing into data centers is not used for AI, according to Nvidia. Much of it gets used up by cooling systems that keep servers from overheating and by electricity traveling long distances across sprawling campuses. That is amplifying carbon emissions from data center energy use, given that operators are increasingly relying on natural gas and coal-fired plants to power their projects. Microsoft Corp., for example, is considering whether to delay or abandon its ambitious clean-energy targets as it tries to remove hurdles that could hold it back in the AI race, people familiar with the matter told Bloomberg last month.
As power demand and data centers grow, those energy losses will only mount. But so will the potential gains from any efficiency improvements, said Gartner analyst Tony Harvey. Racks have gone from each having eight GPUs to 72 starting two years ago, requiring around 150 kilowatts of power.
And power demand keeps growing. Rubin, the name of Nvidia’s new GPU and rack system coming out later this year, will eventually need around 300 kilowatts to run, experts say.
Beyond Rubin, the industry is bracing for chips that bring racks closer to 1 megawatt — or enough to power 750 U.S. homes on average.
“At this scale, it makes a big difference,” he added.
Overall, though, it’s unclear to what extent these fixes can curb data center power demand, given how much bigger projects are getting and how many more of them there are.
As they revamp their facilities, AI players are also investing in energy-efficiency startups. Nvidia, whose servers and chips make up 70% of AI hyperscaler spending, according to Bloomberg Intelligence, has poured millions of dollars into Emerald AI, which makes software that helps data centers avoid overtaxing the grid during peak demand. Hyperscalers like Alphabet Inc.’s Google are working on making their AI models more energy efficient.
“This is a constant pursuit of finding every ounce of efficiency that we can sort of squeeze out of that power envelope,” said Dion Harris, Nvidia’s senior director of high performance computing and AI hyperscale infrastructure solutions.
This exponential growth in capacity has already led to some new data center designs. Nvidia’s Blackwell chip, which it released in 2024, increased processing capacity while using the same amount of energy as its predecessor — representing a leap in energy efficiency.
But it also generated a lot more heat, too much for traditional air cooling systems to keep it from malfunctioning.
Constantly running the air-cooling cycle requires a lot of energy, so companies developed a direct-to-chip liquid cooling method.
Liquid cooling can increase energy efficiency in a data center by 15%, according to a study done by Nvidia and power equipment maker Vertiv Holdings Co.
Now, the AI industry, led by Nvidia, is trying to save energy by streamlining the path of electricity from grid to chip.
Data centers step down the voltage of grid power from 34,500 volts — the dangerously high voltage levels that travel over transmission lines — to the 12 volts chips need.
These conversions take many steps, with energy escaping as heat each time.
“All of those steps introduce little inefficiencies,” said Nvidia’s Harris. “They’re incremental, fractional in a lot of cases, but they add up to large numbers when you’re doing it across a very large campus.”
Nvidia is now trying out a new piece of equipment that consolidates this process into fewer steps, saving energy and space — a sidecar.
An even more radical redesign is in the works for future data centers. The industry is vying to replace some of the electrical room equipment with a solid state transformer — a smarter, electronic device that can switch currents between AC and DC and better handle higher voltages.
The power distribution systems alone account for roughly a third of total power losses, “and that’s to do with all of those transitions,” said Gartner analyst Harvey. “If we can get to that ultimate 800 volts DC, that’ll probably drop to less than 1%.”
Energy saved from a reimagined power system can serve as a climate measure, especially when combined with efforts to use cleaner energy sources. Another potential benefit of switching to a DC power system is that data centers can more easily be connected to renewable energy, which typically generates that kind of current.
“DC power inherently integrates better with renewables,” said Scott Armul, chief product and technology officer at Vertiv.
China, for example, already builds data centers in regions generating excess renewable energy. The U.S. is nowhere near having extra renewable energy, but operators are looking to batteries and solar power to help run data centers, even as they still have to rely on gas power to do most of the heavy lifting.
Power system upgrades are front of mind for AI players. Nvidia, for one, has promised to release new, more powerful chips about once a year. GE Vernova Inc., which makes data center power equipment, is already seeing strong demand from hyperscalers for 800 volt DC systems.
“Everybody is asking us to provide solutions for the next orders to come,” said Philippe Piron, chief executive officer of GE Vernova’s electrification segment.
Forgash and Dottle write for Bloomberg.
The post Inside the race to rebuild AI data centers before the grid hits its limit appeared first on Los Angeles Times.
