On the south side of Austin, Texas, engineers at the semiconductor maker Advanced Micro Devices designed an artificial intelligence chip called MI300 that was released a year ago and is expected to generate more than $5 billion in sales in its first year of release.
Not far away in a north Austin high-rise, designers at Amazon developed a new and faster version of an A.I. chip called Trainium. They then tested the chip in creations including palm-size circuit boards and complex computers the size of two refrigerators.
Those two efforts in the capital of Texas reflect a shift in the rapidly evolving market of A.I. chips, which are perhaps the hottest and most coveted technology of the moment. The industry has long been dominated by Nvidia, which has leveraged its A.I. chips to become a $3 trillion behemoth. For years, others tried to match the company’s chips, which provide enormous computing power for A.I. tasks, but made little progress.
Now the chips that Advanced Micro Devices, known as AMD, and Amazon have created — as well as customer reactions to their technology — are adding to signs that credible alternatives to Nvidia are finally emerging.
For some crucial A.I. tasks, Nvidia’s rivals are proving they can deliver much faster speed, and at prices that are much lower, said Daniel Newman, an analyst at Futurum Group. “That’s what everybody has known is possible, and now we’re starting to see it materialize,” he said.
The shift is being driven by an array of tech companies — from large competitors such as Amazon and AMD to smaller start-ups — which have started tailoring their chips for a particular phase of A.I. development that is becoming increasingly important. That process, called “inferencing,” happens after companies use chips to train A.I. models. It allows them to carry out tasks such as serving up answers with A.I. chatbots.
“The real commercial value comes with inference, and inference is starting to gain scale,” said Cristiano Amon, the chief executive of Qualcomm, a mobile chip maker that plans to use Amazon’s new chips for A.I. tasks. “We’re starting to see the beginning of the change.”
Nvidia’s rivals have also started taking a leaf out of the company’s playbook in another way. They have begun emulating Nvidia’s tactic of building complete computers — and not just the chips — so that customers can wring the maximum power and performance out of the chips for A.I. purposes.
The increased competition was evident on Tuesday, when Amazon announced the availability of computing services based on its new Trainium 2 A.I. chips and testimonials from potential users including Apple. The company also unveiled computers containing either 16 or 64 of the chips, which are combinations that particularly accelerate inferencing performance.
Amazon is even building a kind of giant A.I. factory for the start-up Anthropic, which it has invested in, said Matt Garman, the chief executive of Amazon Web Services. That computing “cluster” will have hundreds of thousands of the new Trainium chips and will be five times more powerful than any that Anthropic has ever used, said Tom Brown, a founder and the chief compute officer of the start-up, which operates the Claude chatbot and is based in San Francisco.
“This means customers will get more intelligence at a lower price and at faster speeds,” Mr. Brown said.
In total, spending on computers without Nvidia chips by data center operators, which provide the computing power needed for A.I. tasks, is expected to grow 49 percent this year to $126 billion, according to Omdia, a market research firm.
Even so, the increased competition does not mean Nvidia is in danger of losing its lead. A spokeswoman for the company pointed to comments made by Jensen Huang, Nvidia’s chief executive, who has said his company has major advantages in A.I. software and inferencing capability. Mr. Huang has added that demand is torrid for the company’s new Blackwell A.I. chips, which he says perform many more calculations per watt of energy used, despite an increase in the power they need to operate.
“Our total cost of ownership is so good that even when the competitor’s chips are free, it’s not cheap enough,” Mr. Huang said in a speech at Stanford University this year.
The changing A.I. chip market has partly been propelled by well-funded start-ups such as SambaNova Systems, Groq and Cerebras Systems, which have lately claimed big speed advantages in inferencing, with lower prices and power consumption. Nvidia’s current chips can cost as much as $15,000 each, and its Blackwell chips are expected to cost tens of thousands of dollars each.
That has pushed some customers toward alternatives. Dan Stanzione, the executive director of the Texas Advanced Computing Center, a research center, said the organization planned to buy a Blackwell-based supercomputer next year, but would most likely also use chips from SambaNova for inferencing tasks because of their lower power consumption and pricing.
“That stuff is just too expensive,” he said of Nvidia’s chips.
AMD said it expected to target Nvidia’s Blackwell chips with its own new A.I. chips arriving next year. In the company’s Austin labs, where it exhaustively tests A.I. chips, executives said inferencing performance was a major selling point. One customer is Meta, the owner of Facebook and Instagram, which says it has trained a new A.I. model, called Llama 3.1 405B, using Nvidia chips but that it uses AMD MI300s chips for providing answers to users.
Amazon, Google, Microsoft and Meta are also designing their own A.I. chips to speed up specific computing chores and achieve lower costs, while still building big clusters of machines powered by Nvidia’s chips. In December, Google plans to begin selling services based on a sixth generation of internally developed chips, called Trillium, which is nearly five times faster than its predecessor.
Amazon, sometimes seen as a laggard in A.I., seems particularly determined to catch up. The company allocated $75 billion this year for A.I. chips and other computing hardware, among other capital spending.
At the company’s Austin offices — run by Annapurna Labs, a start-up that it bought in 2015 — engineers previously developed networking chips and general-purpose microprocessors for Amazon Web Services. Its early A.I. chips, including the first version of Trainium, did not gain much market traction.
Amazon is far more optimistic about the new Trainium 2 chips, which are four times faster than previous chips. On Tuesday, the company also announced plans for another chip, Trainium 3, which was set to be even more powerful.
Eiso Kant, the chief technology officer of Poolside, an A.I. start-up in Paris, estimated that Trainium 2 would provide a 40 percent improvement in computing performance per dollar compared with Nvidia-based hardware.
Amazon also plans to offer Trainium-based services in data centers across the world, Mr. Kant added, which helps with inferencing tasks.
“The reality is, in my business, I don’t care what silicon is underneath,” he said. “What I care about is that I get the best price-performance and that I can get it to the end user.”
The post The Furious Contest to Unseat Nvidia as King of A.I. Chips appeared first on New York Times.
