For three years, Jensen Huang, the chief executive of Nvidia, has described his company’s chips as the Swiss Army knife of artificial intelligence. They were an all-purpose tool ideal for building and running A.I.
But on Monday, in a packed arena for Nvidia’s developer conference GTC in San Jose, Calif., Mr. Huang told the story of an industry with changing needs and how his company, the most valuable publicly traded company in the world, is trying to change with it.
Mr. Huang unveiled a new product incorporating technology from a start-up called Groq. The product will pair Nvidia’s chips, which excel at receiving an A.I. request, with Groq’s chips, which have components that can put a charge into how Nvidia’s chips operate.
Over the past year, A.I. companies have shifted their work. The A.I. systems they built using Nvidia’s chips have improved at creating software code, doing research and making images and videos. These capabilities, the result of a process known as inference, have put more value on chips that can generate data as inexpensively and quickly as possible.
When it comes to cost and speed, Nvidia’s chips have lagged behind those from Google, which makes its own chips called tensor processing units; and upstarts like Cerebras, whose chips specialize in running A.I. The edge those rivals enjoy in inference has helped them win business from some of Nvidia’s longtime customers, like OpenAI and Meta.
The deals caught Mr. Huang’s attention. As competitors began to make a dent into his company’s business last year, Nvidia in December announced a $20 billion licensing agreement with Groq, which makes chips custom-built for inference. With the combined technology, Nvidia is expected to make inference quicker and less expensive, analysts said.
“A.I. is able to do productive work, and therefore the inflection point of inference has arrived,” Mr. Huang said in a keynote speech.
The rush to get the product to market over the past three months speaks to how quickly the A.I. market is changing and how far Nvidia is willing to go to remain the world’s leading chipmaker. In just three years, Nvidia has become a driving force for the U.S. economy because its chips account for more than 90 percent of the A.I. market. It’s a dominant position that Mr. Huang doesn’t want to surrender.
“They’re going to thread this together to protect their moat,” said Daniel Newman, the chief executive of Futurum Group, a tech analysis firm, of the new product.
Nvidia’s annual GTC event has become the hottest ticket in Silicon Valley. The company’s neon green has blanketed the downtown of San Jose. Attendees started queuing hours before Mr. Huang’s keynote speech was scheduled to start. And over the next few days, everyone from start-up founders to major corporate executives will file in and out of 1,000 talks.
The A.I. industry is convening this year as the technology it has been developing has achieved an inflection point. In the early years of generative A.I., there was a divide between developers who said the technology would change everything and users who found it wasn’t that useful.
But last year, that started to shift as A.I. tools from Anthropic, OpenAI and others made it easier to write software code. They also introduced personal digital assistants called agents, which could do tasks on their own.
These breakthroughs have caused people to ask A.I. to do more work. Each request goes to chips in a data center that must create tokens — a term for a chunk of data — that fulfill those demands by responding with code, research or whatever a user wants.
“The number of tokens that are being generated is really, really gone exponential,” Mr. Huang said during a call with analysts last month. “So we need to do inference at a much higher speed.”
Nvidia has already been rewarded for its plans to address the growing demand for inference. Last month, OpenAI said it had struck an agreement with Nvidia for chips with “dedicated inference capacity.”
(The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied the suit’s claims.)
Nvidia’s ability to win over other customers will determine its share of the inference market, Mr. Newman said. He expects the company to retain its 90 percent share of chips for developing A.I., but predicts that it will claim only about a third of chips used to run that A.I. That won’t change Nvidia’s revenue prospects, Mr. Newman said, but it will influence its products.
Nvidia’s deal with Groq also helps it with manufacturing problems that are constraining how fast its sales can grow, said Umesh Padval, a managing partner at the investment firm Seligman Ventures.
Groq’s chips are made by Samsung Electronics, not Taiwan Semiconductor Manufacturing Company, which makes most of Nvidia’s chips and is struggling to meet the company’s demand, Mr. Padval said. And unlike Nvidia’s chips, Groq’s chips don’t require high-bandwidth memory chips. Those chip manufacturers have also been swamped with orders.
“It’s a brilliant supply-chain move,” Mr. Padval said.
Don Clark contributed reporting.
Kalley Huang is a Times reporter in San Francisco, covering Apple and the technology industry.
The post Nvidia Built the A.I. Era. Now It Has to Defend It. appeared first on New York Times.
