New cloud usage data show that as companies pour billions into AI infrastructure, much of that computing power sits idle.
Cast AI, an automated cloud cost optimization and management platform that works with companies like BMW and Cisco, published data in its 2026 State of Kubernetes Optimization Report showing that, on average, most organizations provide about 20 times more GPU capacity than they actively use at any given moment.
According to data collected from 23,000 clusters across thousands of companies using Cast AI’s AI agent, average GPU utilization across enterprise servers is at 5%, meaning roughly 95% of provisioned GPU capacity is not being used. CPU utilization, the report says, is similarly low, at 8% of total capacity.
The report notes that a CPU, which handles basic functions like opening an app, may waste only a few cents per hour when idle, while an unused GPU that handles machine learning, simulations, and video processing would waste several dollars per hour. GPUs can be up to 50 times more expensive than comparable CPU-based machines, the report said.
The findings come as companies race to secure scarce and expensive AI chips, especially for premium GPUs like Nvidia’s Blackwell chips, which are seeing prices rise as AI demand outpaces supply.
Cast AI CEO Laurent Gil told Business Insider that the sense of urgency is part of the problem.
Unlike traditional cloud computing, where resources can be spun up and down on demand, said Gil, companies often commit to long-term GPU contracts due to limited supply. That leads to overbuying driven by the fear of missing out.
“The act of buying has no correlation with whether you need them or not,” Gil said. “You don’t buy them because you need them. You buy them because they were available.”
According to Cast AI’s report, a healthy GPU utilization rate is around 50%.
“I want the CTOs to ask their teams, ‘Hey, we already have a few thousand of those GPUs. How are we using them?'” Gill added. “And if only 5% is being used, before you buy new machines, you have another 20 times more that you didn’t know about available in your account.”
Read the original article on Business Insider
The post Companies are hoarding AI compute because of FOMO — and they’re sitting on most of it appeared first on Business Insider.
