An AI hallucination is output from a generative AI system that sounds plausible but is false or unsupported by its sources; NIST's Generative AI Profile (AI 600-1) calls the same risk 'confabulation.' It happens because a large language model generates text by predicting likely next words, not by retrieving verified records, so it can produce invented citations, numbers or quotes with the same fluency as true ones. A 2025 OpenAI paper (Kalai et al.) argues that common training and scoring methods reward a confident guess over 'I don't know.' The best-known real-world case is Mata v. Avianca (2023), where a federal judge in New York sanctioned two lawyers $5,000 for filing a brief that cited court cases ChatGPT had invented. Grounding answers in retrieved documents reduces hallucination but does not eliminate it, so anything that matters still needs checking against a primary source.
Reading time
— 5 min
Updated
— Sep 27, 2026
Fact-reviewed
— Sep 27, 2026
Key Takeaways
Key Takeaways
1An AI hallucination is a confident, fluent answer that's false or not backed by any source, like a made-up court case or a statistic nobody published.
2It happens because chatbots predict likely words rather than look facts up, so they sound equally sure when they're right and when they're wrong.
3Grounding answers in real documents cuts the risk but doesn't remove it. Anything that matters still needs a check against a primary source.
The concept
Ask a chatbot for three books on a niche topic and it may give you three titles, three authors and three publication years, all perfectly formatted, and one of the books may not exist. That's a hallucination. The chatbot isn't lying on purpose. It writes by predicting which words usually come next, and a book recommendation has a very predictable shape: title, author, year. It can fill that shape with something that sounds right without ever checking a library. The tricky part is the tone. A made-up answer sounds exactly as confident as a correct one, so you can't tell them apart by how they read.
This is the practical difference between AI and traditional software: a calculator or database either returns the stored answer or an error. A language model always returns something. The stakes rise with AI agents, which act on their outputs, so a made-up detail can become a wrong action, not just a wrong sentence.
Quick check
Why does a hallucinated answer usually sound so convincing?
Worked examples
Example 1: The invented court cases (baseline case)
In Mata v. Avianca, a personal-injury suit against an airline in federal court in New York, the plaintiff's lawyers filed a brief in 2023 citing decisions such as "Varghese v. China Southern Airlines." Opposing counsel and the judge couldn't find them, because ChatGPT had generated them, complete with quotes and docket-style numbers. When asked, one lawyer had even asked ChatGPT whether the cases were real, and it said yes. Judge P. Kevin Castel sanctioned the two lawyers and their firm $5,000. The lesson isn't that AI can't help with research. It's that asking the model to confirm its own output is not verification.
Example 2: The summary that adds a detail (edge case)
You paste in a 2-page meeting note and ask for a summary. The summary is accurate except for one line: "The team agreed to launch on March 3." The note mentions March but never a specific date. That's an extrinsic hallucination: the model filled in a detail that fits the pattern of meeting summaries. It's harder to catch than an invented book because 95% of the output is correct and you gave it the source. Spot-check any date, number or name in a summary against the original.
Example 3: Using a chatbot safely for a real task (real-world use)
You want to know the return policy for a laptop you bought. Asking a general chatbot "What's Brand X's return window?" invites a guess: it may give a common number like 30 days that doesn't match your retailer or country. A safer workflow is to open the retailer's actual policy page, paste the text in, and ask the chatbot to answer only from that text and quote the sentence it relied on. Then read that sentence yourself. Two minutes of checking turns a plausible answer into a verified one.
Quick check
In Mata v. Avianca, why wasn't asking ChatGPT 'are these cases real?' enough?
How it works (visual)
How an AI hallucination happens
Common mistakes
Common Mistakes
✕
Treating a confident tone as a sign of accuracy.
→ Judge the answer by whether you can trace it to a real source, not by how sure it sounds.
✕
Asking the chatbot to double-check itself and stopping there.
→ Check outside the chat: the original document, an official site, or a database built for that purpose.
✕
Using a general chatbot for citations, legal cases, medical doses or exact figures.
→ Use it to draft or explain, then pull the exact facts from primary sources.
Common misconception
“AI hallucinations are rare glitches that newer models have fixed.”
They've become less frequent as models improved and as tools add web search and document retrieval, but they come from how the technology generates text, so they haven't disappeared. They're still most likely on rare facts, precise numbers, citations and recent events, which are exactly the details people most want to copy into their work. Fluent, confident text also means you can't judge where a passage came from by how it reads; how to tell if content was AI-generated covers what actually works.
What to do next
What to do next
Next time a chatbot gives you a number, name or citation you plan to reuse, find it in the original source first.
When you have the source document, paste it in and ask the tool to answer only from it and quote its evidence.
Ask the chatbot to say when it isn't sure. It won't catch everything, but it helps.
Keep AI for drafting, brainstorming and explaining; keep primary sources for facts that carry consequences.
FAQ
FAQ
Related terms
Related terms
Hallucination
A generative AI output that sounds plausible but is false or not supported by any source.
Confabulation
NIST's term for the same risk: the system confidently produces made-up content, much as a person might fill a memory gap without knowing it.
Grounding
Tying a model's answer to specific retrieved documents or data, so claims can be traced and checked.
Retrieval-augmented generation (RAG)
A setup where the system first searches a document collection, then writes its answer using what it found.
Calibration
How well a system's confidence matches how often it's actually right.
This entry was researched from public sources and drafted with AI-assisted tools, then edited — errors are still possible. Spot one, or want a topic covered? Read our disclaimer.
KnowThis