FACT CHECK: How Trustworthy Are AI Fact-Checks from AI Chatbots Like Grok and ChatGPT?

Claim:

AI chatbots like Grok, ChatGPT, Gemini, Copilot, and others are reliable tools for fact-checking news and information.

Fact:

AI chatbots often produce confident but incorrect responses, fabricate quotes, struggle with news context, and are not fully reliable for accurate fact-checking.

What’s the Context?

Ever since Elon Musk’s xAI launched the AI chatbot Grok in November 2023, many users on X (formerly Twitter) have been tagging it with “Hey, @Grok, is this true?” to rapidly fact-check trending posts. Following its rollout to non-premium users in December 2024, AI-based fact-checking became even more accessible.

At the same time, a TechRadar survey found that 27% of Americans now turn to tools like ChatGPT, Meta AI, Google’s Gemini, Microsoft’s Copilot, or Perplexity for information—bypassing traditional search engines like Google or Yahoo. But how trustworthy are these AI responses?

What the Evidence Says

Accuracy Problems Identified

Two major studies in 2024 from BBC and the Tow Center for Digital Journalism reveal alarming inaccuracies in AI chatbot outputs:

• The BBC found that 51% of responses by ChatGPT, Copilot, Gemini, and Perplexity based on BBC articles had significant issues.
  • 19% contained fabricated facts
  • 13% had altered or missing quotes
• The Tow Center study, published in the Columbia Journalism Review, found that Grok answered 94% of provenance queries incorrectly, with Perplexity performing best (still with a 37% failure rate).

AI Answers with ‘Alarming Confidence’

AI models like ChatGPT often present wrong answers with high confidence, rarely admitting uncertainty or refusing to respond. This misleading certainty increases the risk of misinformation.

High-Profile Failures by Grok and Others

• White Genocide Controversy: Grok repeatedly brought up the conspiracy theory of “white genocide” in South Africa—even when unrelated questions were asked. xAI later blamed an “unauthorized modification.”
• Basketball ‘Bricks’ Joke Misfire: Grok interpreted a joke about a basketball player “throwing bricks” (missing shots) as a criminal vandalism case.
• Biden Ballot Deadline Misinformation: After Biden’s 2024 withdrawal, Grok falsely stated Kamala Harris would miss ballot deadlines in nine states.
• Fake AI Image Identifications: Grok misidentified an AI-generated fire image, claiming it depicted incidents at multiple real airports, despite its errors (e.g., inverted plane tails).
• ChatGPT’s Personal Lie: Meta AI claimed to be a parent with a disabled child in a Facebook group—only to admit later that it doesn’t have personal experiences.

Why These Errors Happen

AI chatbots pull data from massive text databases scraped from the internet. If these datasets include biased, false, or politically manipulated sources, the answers reflect those flaws.

Bottom Line: Can You Trust AI Fact-Checks?

No — not entirely.
While AI chatbots can be useful for basic fact-checking tasks, their tendency to:

• fabricate quotes and facts,
• misinterpret context,
• cite fake sources, and
• answer confidently without verification

makes them unreliable as standalone fact-checkers.

Verdict: MISLEADING

AI chatbots like Grok and ChatGPT should not be treated as reliable fact-checking tools. Always verify their answers with trusted journalistic sources, government websites, or professional fact-checking organizations like factcheck india.