It’s only been available to the public for less than a week. Still, the latest, lightweight version of Google’s Gemini generative AI, Gemini 3 Flash, has been making headlines today for the wrong reasons.
The cause of its newfound angst is a reported 91 percent hallucination rate, according to an analysis of it and a bunch of other generative AIs, including past and present versions of Gemini, ChatGPT, Claude, and others.
That 91 percent hallucination rate that’s making the rounds on the web today, though, requires a closer look and some clarification. Because it doesn’t mean that Gemini 3 Flash is inaccurate. How can that be? Well, it’s complicated.
Gemini 3 Flash Is Extremely Accurate—Until It Starts Lying
Most of the time, when it knows the answer, Gemini 3 Flash gives a remarkably truthful and accurate answer. And when it’s wrong, it goes off the rails and begins making s—t up like nobody’s business.
What that hallucination rate refers to isn’t the percentage of its answers that are wrong or include false information. It means that in 91 percent of cases where it doesn’t know the answer, it makes one up.
“We measure hallucination rate as how often the model answers incorrectly when it should have refused or admitted to not knowing the answer,” says Artificial Analysis, which performed the study.
It’s how the report could also say, “(Gemini 3 Flash) has the highest knowledge accuracy of any model tested.” It is both the most accurate (when it knows a query’s answer) of all the AIs that Artificial Analysis tested, and the most likely to hallucinate and make up facts when it doesn’t know the answer.
According to Google, Gemini 3 Flash is as fast at dishing out answers as the traditional Google search engine. And it’s still an impressive feat. I’ve argued that rather than being chatbots, a more compelling use for AIs is as a search engine replacement, or at least a supplement.
The big downside is how much longer it takes to get an answer from one. But if Gemini can do it almost as fast, that’s one significant barrier knocked down. If only it could be more honest and say “I don’t know” more often, then Google would have something really special on its hands.
The post Google’s Gemini 3 Flash Is Smart, Fast, and Weirdly Dishonest appeared first on VICE.
