Analysis: Gemini 3-based AI Overviews are accurate ~90% of the time — that still leaves tens of millions of errors every hour

The finding

It has been reported that the New York Times analyzed Google’s new AI Overviews — the short, generative summaries that sit atop search results and are powered by Gemini 3 — and found they are accurate roughly 90% of the time. Sounds reassuring at first glance. But scale is a cruel magnifier. With Google handling roughly 5 trillion searches a year, it has been reported that even a 10% error rate translates to tens of millions of erroneous answers rolling out every hour.

Why this matters

Accuracy percentages are one thing. Real-world harm is another. When a fast, authoritative-seeming summary gets a fact wrong, people act on it — journalists, students, shoppers, patients, voters. The emotional hit is trust: once you catch a high-profile mistake, can you un-see the next one? The NYT’s analysis suggests this is not a niche problem but a systemic one, amplified by sheer volume. Ask yourself: would you rather a human editor or an algorithm be your first stop for tricky, consequential questions?

The broader context

This story lands in the middle of a larger debate about generative AI in search. Tech companies promise convenience and speed; critics warn about hallucinations and overconfidence. Regulators and competitors are watching closely. It has been reported that Google’s system sometimes mixes facts or invents sources — a classic generative-AI pitfall — and that fixing it will require both better models and clearer product design so users know when to double‑check.

The takeaway

Numbers tell a blunt truth: high accuracy percentages can still mean a lot of mistakes if your user base is massive. The challenge now is operational: improve model reliability, surface provenance, and build guardrails that slow down the most consequential errors. Because convenience is fine — until it costs you your credibility.

Sources: nytimes.com