Google has been in the AI chip game for more than a decade. Now its custom hardware is moving markets.
Shares of Nvidia and other chipmakers tumbled last month following a report that Meta — one of Nvidia’s largest customers — was exploring a deal to use Google’s AI chips, known as Tensor Processing Units, or TPUs.
Google has primarily used its TPUs for internal use, but it also leases them to external customers through the cloud. Nvidia, meanwhile, has become the dominant provider of AI chips with its graphics processing units, or GPUs.
Google has a potential blockbuster business to unlock. In a research note sent December 2, Morgan Stanley projected that 5 million of Google’s TPUs will be purchased in 2027 and about 7 million in 2028, significantly increasing its prior projections.
Here’s a breakdown of everything you need to know about TPUs, what they’re used for, and when they might become a more prominent threat to Nvidia’s chip dominance.
What are TPUs?
Over a decade ago, Google needed more powerful and specialized compute power for the type of AI work it wanted to do. A team led by the now-CEO of Groq, Jonathan Ross, designed a new chip based on a specific type of integrated circuit for machine learning. The TPU was born.
Google has continued to improve and refine its TPUs over the years, making them more effective at both training models and inference, the process where a trained model answers a question or performs a task. As large language models have grown in size, Google has also increased the memory bandwidth of later TPUs to handle these bigger workloads.
Google says its latest “Ironwood” TPU, which the company is making widely available, is more than four times better than its predecessor for both training and inference.
How are TPUs different from Nvidia’s GPUs?
Nvidia’s GPU cards, which launched in 1999, were originally designed for gaming, not AI. When researchers later discovered that the chips were useful for tasks such as training neural networks, Nvidia doubled down on the AI market.
Google’s TPUs, on the other hand, were designed for AI from the get-go. Being more specialized means they’re more efficient than Nvidia’s chips at some tasks, and faster for running select AI models. They have something called a systolic array, which lets a more constant stream of data pass through the chip, rather than having to keep fetching more from memory.
Where Google’s TPUs have a big edge is in cost at scale. It’s possible to have thousands of TPUs working in tandem in a single “pod.” Because they are faster at some calculations than GPUs, running many TPUs at once can sometimes be more cost-efficient.
That may become more important as companies increase investments at the inference stage, which Google says its latest TPU is especially good at. That could save companies money.
What’s stopping customers from swapping GPUs for TPUs?
One advantage that Nvidia has over Google is its CUDA software, which makes it possible for regular applications to use GPUs for general computing tasks — not just for graphics. CUDA also only works with Nvidia chips, one of the biggest points of friction stopping companies from switching to Google’s chips.
Google is trying to change that. For example, there is a lot of industry demand to have TPUs better support Pytorch, a popular tool for building AI applications that was created inside Meta. Google is allocating more resources and attention to better supporting Pytorch, which data suggests is seeing much more demand than Google’s own Tensorflow software, according to Google employees and industry experts.
Who uses TPUs?
When it comes to TPUs, Google remains its own biggest customer. It uses the chips across the company to power products like Search and Maps. Its latest Gemini 3 model was trained on TPUs.
While Google has prioritized its own needs, it has leased its TPUs to other customers. Apple used TPUs to train its in-house AI model, Business Insider previously reported. In October, Anthropic announced a blockbuster deal with Google that it says will see it using up to 1 million TPUs. Broadcom, which helps build the TPU chips for Google, revealed in its Q4 earnings call this week that it has received a total of $21 billion in orders from Anthropic for Google’s Ironwood TPUs.
Meta is also in early testing of Google’s TPUs, according to a person familiar with the matter, although it’s not clear if it will result in a long-term deal.
What could this mean for Google — and Nvidia?
Google’s TPU business could be poised to explode in the coming years. In a research note this month, Morgan Stanley wrote that every 500,000 TPU chips sold could potentially add around $13 billion in revenue to Google’s balance sheet in 2027.
Having TPUs isn’t just a potential revenue boost for Google. It also gets a feedback loop in using TPUs to run and train its AI models, learning from them, and changing how it develops its next chips to be better at the things Google needs.
Nvidia’s GPU is still very much the chip du jour, but other tech giants are increasingly pouring resources into their own custom chips. Amazon just announced its new Trainium3 custom AI chip, which the company says can cut the cost of training and powering AI models in half compared with a GPU.
Many TPU buyers are also Nvidia customers, and some industry insiders told Business Insider that it’s more likely that the rise of more specialized chips will see companies and labs diversify the chips they use, rather than going all in on one provider.
That could still hurt Nvidia, which could lose pricing power as a result of a more diverse market.
However, even if Google’s TPU business finds more momentum, it doesn’t mean Nvidia will suddenly crumble overnight. Jordan Nanos, member of technical staff of research firm SemiAnalysis, said his firm believes companies, including Nvidia, Google, and Amazon, will all “sell lots” of chips in the future.
“We don’t see TPU as a significant threat to Nvidia’s business, but it has been a real player in the market for many years,” said Nanos. “It is possible that Google sells TPU servers externally in the future, to many more customers. Right now, they are very selective.”
Have something to share? Contact this reporter via email at [email protected] or Signal at hughlangley.01. Use a personal email address and a non-work device; here’s our guide to sharing information securely.
Read the original article on Business Insider
The post What are TPUs? Everything you need to know about Google’s market-moving AI chips. appeared first on Business Insider.
