Sleeping Stanford and Hallucinated Hallucinations
Stanford University’s shaky metrics and the myth of AI hallucinations: A deep dive into why fabricated outputs aren’t 'hallucinations' but 'trajectory drift.'
There are anecdotes of Russian soldiers unfamiliar with Western-style toilets using them incorrectly. For example, in Karolina Lanckorońska’s memoir Michelangelo in Ravensbrück, she recounts an incident duing world war 2 where a Russian officer, unfamiliar with the purpose of a toilet, used it to wash his hair, leading to confusion and accusations of sabotage. These soldiers didn’t hallucinate—they inferred the wrong association between known concepts when encountering a new one. Similarly, AI errors are often misinterpreted as hallucinations when they are better understood as misapplied inferences.
Labels Can Be Misleading
My last blog dealt with the technical foundation of artificial intelligence such as Large Language Models (LLM) and how to put fabricated content they produce into context.
This got me thinking again: We have hallucination which refers to the generation of outputs that are factually incorrect, misleading, or nonsensical, despite appearing coherent and plausible. And this gets everyone up in arms.
But there is also morphing when which can occur in temporal sequences where the AI generates or manipulates frames of a video. Errors can arise when the AI fails to maintain object consistency across frames, causing: Objects to deform, merge, or transform unnaturally. "Morphing artifacts" where features bleed or transition incorrectly (e.g., a person’s face blending into another inappropriately).
Morphing and Hallucination are ultimately caused by a spatial misalignment in the latent space encoding an AIs knowledge. They stem from latent space errors, where AI systems misalign or drift within their internal representation of data. But the term hallucination invokes debates about the social issues and uncontrollable risk when morphing receives much less attention. Trajectory drift (morphing) can lead to visually inconsistent or illogical scenes. For sensitive applications like medical imaging or autonomous vehicles, this is just as critical as hallucination in still images. And this points to a problem. The term hallucination has become problematic and a distraction.
But no problem I have a solution
Hallucinated "AI Hallucination" and How to Stop It
This paper critiques the ontological misclassification of AI outputs as "hallucinations" and proposes the term "trajectory drift" as a more precise descriptor for errors arising from latent space misalignments. By reframing the concept of fabricated outputs as trajectory drift, we highlight the nuanced challenges inherent to probabilistic systems while addressing the need for clearer terminology to improve both technical solutions and public understanding.
Keywords: LLM, hallucination, trajectory drift, artificial intelligence
Suggested Citation:
Werner, Swen, Hallucinated "AI Hallucination" and How to Stop It (December 19, 2024). Available at SSRN: https://ssrn.com/abstract=5063992
The recent HA Index from Stanford University covers this topic under the category of “Factuality and Truthfulness.” It refers to the HaluEval Benchmark, which suggests that 19.5% of ChatGPT’s generated content is fabricated which I covered before.
This research which this number is based on is rather questionable. The methodology chosen by this study is unsuitable to confidently draw such conclusions and reveals a misunderstanding of how AI technically operates.
Consider this example: ChatGPT was asked to generate a list of five important dates in U.S. history. Its response included:
July 4, 1776 - Declaration of Independence signing
The actual signing, however, began on August 2, 1776, while July 4 marks the adoption of the Declaration. This was marked down as an example of hallucination. The model’s choice of July 4 reflects cultural prominence in its training data, not an arbitrary invention. Similarly, queries like “Who took the first picture of a cat?”—a highly esoteric question—blur the line between probabilistic synthesis and factual retrieval. The AI generates plausible answers based on learned patterns but is not optimized for deterministic fact-checking.
Hi Stanford University, are you actually reading the source material you’re referencing, or are you on autopilot? It is shocking and unacceptable to give such flawed material a platform.
This behaviour of AI simply illustrates the inherent trade-off in probabilistic systems: they prioritise what is most statistically likely rather than committing to deterministic accuracy. Labelling this as a hallucination misunderstands the model’s purpose.
Evaluations like HaluEval often misclassify statistical generalizations as hallucinations. For example, asking an LLM to retrieve obscure trivia such as historical dates or the origins of niche cultural phenomena conflates its strengths (synthesis) with tasks better suited to structured databases. This conflation risks misdirecting both research and public understanding. It is a big issue I think.
Moreover, studies attributing hallucination to architectural flaws or attention mechanisms often overstate their role. And I read more than enough “research” on this topic in last 24 hours to get a good idea that there is a lot of flawed thinking on this topic. While issues like improper weighting or incomplete input encoding can lead to specific errors, these explanations fail to account for the probabilistic nature of language models. Instead, many so-called hallucinations are better explained by ambiguous prompts, gaps in training data, or overgeneralizations.
Why “Trajectory Drift” Is a Better Term
I propose the term “trajectory drift” to describe the latent space misalignments that cause erroneous outputs in AI systems. This term provides:
Mathematical Precision: “Trajectory drift”; reflects the behaviour of latent space vectors, aligning with the underlying mechanics of generative models.
Clarity: It distinguishes between gradual drift (subtle feature changes) and sudden drift (abrupt collapses), which “morphing”; fails to capture.
Diagnostic Power: Framing errors as drift allows researchers to identify specific failure points and refine training or architecture.
For example, gradual drift might occur when an AI slowly misaligns features over a sequence of frames, leading to subtle distortions in a generated video. In contrast, sudden drift describes abrupt and dramatic misalignments, such as complete scene collapses in video generation.
Why Terminology Matters
Defining fabricated outputs as trajectory drift makes it clear that these errors are a function of probabilistic systems rather than failures of perception. It highlights that this effect cannot be entirely eliminated but can be mitigated with better alignment and training. Moving away from anthropomorphic terms also reduces unnecessary fear and clarifies the strengths and limitations of AI systems.
Furthermore, distinguishing adaptive tasks (e.g., creative text generation) from static tasks (e.g., coding or historical retrieval) ensures appropriate deployment of LLMs. For instance:
Adaptive tasks: Require probabilistic flexibility, where variability and creativity are strengths.
Static tasks: Require deterministic precision, where grounding in external knowledge bases is necessary.
Conclusion
Adopting the terminology of trajectory drift repositions the discourse around AI errors. It provides researchers with a clear framework for diagnosing and addressing latent space misalignments while demystifying AI’s limitations for the public and policymakers. By addressing errors at their root and refining use cases, we can advance both the reliability of AI systems and the clarity of discussions surrounding them.