By Stuart Kerr, Technology Correspondent
Published: September 2025 | Updated: September 2025
Contact: [email protected] | @LiveAIWire
AI’s rise has been hailed as the dawn of intelligence beyond human reach. Yet, beneath the headlines lies a provocative counter-argument: what if today’s systems are not truly intelligent at all? Evolutionary biologist David Krakauer has described much of AI as “fake intelligent,” likening its output to “students copying answers from a library” rather than reasoning independently. This tension between appearance and understanding raises questions about how we define intelligence in the age of machines.
The Illusion of Comprehension
Large language models can generate fluent text, diagnose medical scans, and even write software. Yet their methods rely on statistical prediction, not comprehension. Scholars such as Krakauer and Melanie Mitchell argue in an academic study that these systems produce words that look right without knowing why they are right. This idea echoes the critique of “stochastic parrots,” a metaphor capturing how AIs may only repeat patterns found in training data rather than developing real insights, as explored in a Santa Fe Institute essay.
The question, then, is whether predictive brilliance equates to thinking. If an algorithm can generate convincing answers but has no concept of truth, does that qualify as intelligence? Or is it an elaborate mirror reflecting back fragments of our own knowledge? At LiveAIWire, we’ve seen this debate surface in contexts as varied as environmental impact, highlighted in Beyond Algorithms — Hidden Carbon & Water, and in the growing challenges of content verification.
Tools That Compete with Cognition
Krakauer distinguishes between “complementary” cognitive artifacts that extend human ability and “competitive” ones that risk replacing it. The printing press expanded literacy; AI threatens to outsource reasoning itself. Critics such as John Danaher have warned of “competitive cognitive artifacts” that may erode rather than amplify our judgement.
This concern is no longer theoretical. As search engines integrate generative summaries, the issue of whether machines understand or merely present polished approximations becomes a live question for publishers and readers alike, a tension explored in Can Publishers Survive Zero‑Click Era?. What happens when we start trusting answers that lack genuine grounding in comprehension?
Learning Like Humans, or Merely Copying?
Some researchers propose raising AI systems as if they were children—allowing them to acquire “core knowledge” gradually. A recent academic paper highlights how large language models may lack the foundational structures humans use to interpret the world. Without such scaffolding, their outputs risk being clever mimicry rather than authentic reasoning.
This gap becomes pressing in sensitive domains. When generative AI is deployed in classrooms, as LiveAIWire has explored in Generative AI in the Classroom, the distinction between imitation and understanding is crucial. Students may absorb confident but shallow answers, mistaking fluency for knowledge—a danger that mirrors AI’s own blind spots.
The Social and Ethical Stakes
The critique of “fake intelligence” is not mere semantics. If we overestimate what machines understand, we risk underestimating the importance of human oversight. Krakauer warns in the Santa Fe Institute essay that ceding too much cognitive ground to AI could diminish our capacity for independent judgement. The ethical stakes grow sharper when AI is used to manipulate emotions, influence behaviour, or shape public opinion, as examined in AI and Emotional Manipulation.
Ultimately, the question is not just whether AI can generate outputs but whether it can think. And if it cannot, how should society calibrate its trust? Recognising the limits of today’s systems is the first step toward building technologies that support, rather than supplant, human judgement.
About the Author
Stuart Kerr is the Technology Correspondent for LiveAIWire. He writes about artificial intelligence, ethics, and how technology is reshaping everyday life. Read more