Meta’s Llama has reached a turning point with developers as delays and disappointment mount

At LlamaCon, Meta’s first-ever conference focused on its open-source large language models held last month, developers were left wanting.

Several of them told Business Insider they expected a reasoning model to be announced at the inaugural event and would have even settled for a traditional model that can beat alternatives like DeepSeek’s V3 and Qwen, a group of models built by Alibaba’s cloud firm.

A month earlier, Meta released the fourth generation of its Llama family of LLMs, including two open-weight models: Llama 4 Scout and Llama 4 Maverick. Scout is designed to run on a single graphics processing unit, but with the performance of a larger model, and Maverick is a larger version meant to compete with other foundation models.

Alongside Scout and Maverick, Meta also previewed Llama 4 Behemoth, a much larger “teacher model” still in training. It is designed for distillation, which enables the creation of smaller, specialized models from a larger one.

The Wall Street Journal reported on Thursday that Behemoth would be delayed, and that the entire suite of models was struggling to compete. Meta said these models achieve state-of-the-art performance.

Meta‘s Llama used to be a big deal. But now it’s sliding farther down the AI world’s leaderboards, and to some, its relevance is fading.

“It would be exciting if they were beating Qwen and DeepSeek,” Vineeth Sai Varikuntla, a developer working on medical AI applications, told BI at the conference last month. “Qwen is ahead, way ahead of what they are doing in general use cases and reasoning.”

The disappointment reflected a growing sense among some developers and industry observers that Meta’s once-exciting open-source models are losing momentum, both in technical performance and developer mindshare.

While Meta continues to tout its commitment to openness, ecosystem-building, and innovation, rivals like DeepSeek, Qwen, and OpenAI are setting a faster pace in areas like reasoning, tool use, and real-world deployment.

Meta aimed to reassert its leadership in open-source AI. Instead, it raised fresh questions about whether Llama is keeping up.

“We’re constantly listening to feedback from the developer community and our partners to make our models and features even better and look forward to working with the community to continue iterating and unlocking their value,” Meta spokesperson Ryan Daniels told BI.

A promising start

In 2023, Nvidia CEO Jensen Huang called the launch of Llama 2 “probably the biggest event in AI” that year. By July 2024, the release of Llama 3 was held up as a breakthrough — the first open large language model that could compete with OpenAI.

Llama 3 created an immediate surge in demand for computing power, SemiAnalysis Chief Analyst Dylan Patel told BI at the time.”The moment Meta’s new model was released, there was a big shift. Prices went up for renting GPUs.”

Google searches containing “Meta” and “Llama” similarly peaked in late July 2024.

Llama 3 was an American-made, open, top-of-the-line LLM. Though Llama never consistently topped the leaderboard on industry benchmarks, it’s traditionally been influential — relevant.

But that has started to change.

The models introduced a new-to-Meta architecture called “mixture of experts,” which was popularized by China’s DeepSeek.

The architecture allows the model to activate only the most relevant expertise for a given task, making a large model function more efficiently, like a smaller one.

Llama 4’s debut quickly met criticism when developers noticed that the version Meta used for public benchmarking was not the same version available for download and deployment. This prompted accusations that Meta was gaming the leaderboard. The company denied this, saying the variant in question was experimental and that evaluating multiple versions of a model is standard practice.

While competing models paced out ahead, Meta looked rudderless.

“It did seem like a bit of a marketing push for Llama,” said Mahesh Sathiamoorthy, cofounder of Bespoke Labs, a Mountain View-based startup that creates AI tools for data curation and training LLMs, previously told BI.

There’s no singular resource that can measure which model or family of models is winning with developers. But what data exists shows Llama’s latest models aren’t among the leaders.

Qwen, in particular, hovers around the top of leaderboards across the internet.

Artificial Analysis is a site that ranks models based on performance, and when it comes to intelligence, it places Llama 4 Maverick and Scout just above OpenAI’s GPT-4 model, released at the end of last year, and below xAI’s Grok and Anthropic’s Claude.

Openrouter offers a platform for developers to access various models and then publishes leaderboards for model use through its own API. It shows Lama 3.3 among the top 20 models used as of early May, but not Llama 4.

“They wanted to cast a wider net and appeal to enterprises, but I think the technical community was looking for more substantial model improvements,” Sathiamoorthy said.

More than a model

The standard evaluations of Llama 4 released to the public were lackluster, according to experts.

But the muted enthusiasm for Llama 4, compared to Llama 3, goes beyond the model itself, AJ Kourabi, an analyst at SemiAnalysis focused on models, told BI.

“Sometimes it’s not the evals that necessarily matter. It’s the tool-calling and capability for the model to extend beyond just being a chatbot,” Kourabi said.

“Tool-calling” is a model’s ability to access and instruct other applications on the internet or on a user’s computer or device. It’s essential for agentic AI, which promises to eventually book our airline tickets and file our work expenses.

Meta told BI that Llama models support tool-calling, including through its API in preview.

Theo Browne, a YouTuber and developer whose company, Ping, builds AI software for other developers, told BI that tool-calling is increasingly important as agentic tools are coming into focus, and it is almost a requirement for cutting-edge relevance.

Anthropic was an early leader in this, and other proprietary models like OpenAI are catching up, Browne said.

“Having a model that will reliably call the right tool to get the right information to generate the right response is incredibly valuable, and OpenAI went from kind of ignoring this to seemingly being all in on tools,” Browne said.

Kourabi says the biggest indicator that Meta has fallen behind is the absence of a reasoning model, perhaps an even more fundamental element in the agentic AI equation.

“The reasoning model is the main thing, because when we think about what has unlocked a lot of these agentic capabilities, it’s the ability to reason through a specific task, and to decide what to do,” he said.

Llama: Who is it good for?

Some see Llama 4 as evidence that Meta is falling behind, but like Meta’s foundational product, Facebook, AI practitioners say, it’s still almost impossible to write it off.

Nate Jones, the head of product at RockerBox, offers advice to young developers through his Substack, YouTube, and TikTok. He encourages them to put Llama and any other models they’re intimately familiar with on their résumés.

“In 2025, people will already have Llama in their stack and they will look for people who have worked with it,” Jones said.

Paul Baier, the CEO and principal analyst at GAI Insights, consults with companies on AI projects, with an emphasis on non-tech companies. He said Llama is likely to stay in the mix of many, if not most, of his clients.

“Enterprises continue to see that open source is an important part to have in the mix of their intelligence,” Baier told BI. Open models, Llama most prominent among them, can handle less complicated tasks and keep costs down. “They want closed and open,” Baier said.

And that’s what many developers think too, Baris Gultekin, Head of AI at Snowflake, said.

“When our customers evaluate models, they are rarely looking at these benchmarks,” Gultekin said. “Instead, they’ll evaluate these models on their own problem statement. Given the very low cost, Llama is sufficient.”

At Snowflake, Llama powers workloads like summarizing sales call transcripts and extracting structured information from customer reviews. At data platform company Dremio, Llama generates structured query language code and writes marketing emails.

“For 80% of applications, the model probably doesn’t matter,” Tomer Shiran, cofounder and chief product officer at Dremio, told BI. “All the models now are good enough. OpenAI, Llama, Claude, Gemini — they all meet a specific need that the user has.”

Llama may be slipping away from direct competition with the proprietary models, at least for now. But other analysis suggests that the field is diversifying, and Llama’s role in it is solidifying.

Benchmarks are not what drives model choice a lot of the time.

“Everybody’s just testing it on their own use cases,” said Shiran. “It’s the customer’s data, and it’s also going to keep changing.”

Gultekin added: “They usually make these decisions not as a one-time thing, but rather per use case.”

Llama may be losing developers like Browne, who breathlessly await the next toy from a company on the frontier. But the rest of the developer world, the one that’s just trying to make AI-powered tools that work, Llama hasn’t lost them yet. That means Llama’s potential could still be intact.

It’s also part of an open-source playbook Zuckerberg has used since 2013, when the company launched React, a library for building consumer interfaces that’s still in use.

PyTorch is a machine learning framework created in 2016 that overtook Google’s similar effort. Meta transferred PyTorch to the Linux Foundation in 2022 to maintain its neutrality.

“If Meta anchors another successful ecosystem, Meta gets a lot of labor from the open-source community,” RockerBox’s Jones said. “Zuckerberg gets tailwinds that he wouldn’t have had otherwise.”

Have a tip? Contact this reporter via email at [email protected] or Signal at 443-333-9088. Use a personal email address and a nonwork device; here’s our guide to sharing information securely.

Have a tip? Contact this reporter via email at [email protected] or Signal at +1408-905-9124. Use a personal email address and a nonwork device; here’s our guide to sharing information securely.

The post Meta’s Llama has reached a turning point with developers as delays and disappointment mount appeared first on Business Insider.