A student sent me her seminar paper. Three perfectly formatted citations — author, year, publisher, page number. Everything looked correct. Everything was invented. The sources didn't exist. The author was real, but the book was fabricated from scratch.
She wasn't sloppy. She trusted the AI.
This is the real problem with hallucinations. Not that models make up facts — that's expected. But that they output lies with the exact same tone as truths. There's no tone shift. No signal. No raised eyebrow. Just perfect confidence.
The epistemology of hallucinations
Hallucinations are often described as bugs — flaws in the model that need fixing. But from a sociological perspective, they're something else entirely: the consequence of a system that generates plausible text without any connection to a truth condition.
Luhmann would say: the system operates, but it has no reference to the outside world. It processes meaning internally. The question isn't "is this true?" but "does this fit the pattern?"
What that means for teaching
When students use AI for research, we need to teach them a new skill: verification competence. Not just using AI, but doubting it systematically. Every AI output is a hypothesis, not a fact. The burden of proof shifts to the user.
Try it now: Ask an AI to cite three academic sources on any topic. Then check each one. How many exist? How many are hallucinated?