"Fifty-one percent of responses were judged to have “significant issues” in at least one of these areas, the BBC found. Google Gemini fared the worst overall, with significant issues judged in just over 60 percent of responses, while Perplexity performed best, with just over 40 percent showing such issues.

Accuracy ended up being the biggest problem across all four LLMs, with significant issues identified in over 30 percent of responses (with the “some issues” category having significantly more). That includes one in five responses where the AI response incorrectly reproduced “dates, numbers, and factual statements” that were erroneously attributed to BBC sources. And in 13 percent of cases where an LLM quoted from a BBC article directly (eight out of 62), the analysis found those quotes were “either altered from the original source or not present in the cited article.”

Some LLM-generated inaccuracies here were subtle points of fact, such as two responses claiming an energy price cap was “UK-wide,” even though Northern Ireland was exempted. Others were more directly incorrect, such as one that said the NHS “advises people not to start vaping”—the BBC coverage makes clear that the NHS recommends vaping as an effective way to quit smoking."

https://arstechnica.com/ai/2025/02/bbc-finds-significant-inaccuracies-in-over-30-of-ai-produced-news-summaries/