Debates are everywhere about the power and limitation of AI. Some fall into wild hype. Others are too conservative. Here, phenomenologist Harald Wiltsche, and philosopher and lead for Responsible AI at Microsoft Ken Archer, argue that AI is doing something truly powerful and is really intelligent — just not in the way we usually think. Language, maths, logic, and code, allow the structures of thought and meaning to exist outside of the organisms that produce them — and there, they take on a life of their own. AI continues this trajectory, extending the relational power of meaning out into the world outside the human head. 

Most people’s experience of artificial intelligence is one of genuine confusion. They find themselves impressed, sometimes even startled, by what these systems can do, and at the same time, unconvinced that what they are witnessing is really thinking. And their confusion is understandable. On the one hand, we routinely experience AI systems as clarifying our thoughts, drafting arguments, doing our tax reports, even sharpening our own understanding. These are not trivial achievements. They are hard to describe as anything other than intelligent.

On the other hand, these same systems make mistakes that seem almost absurd. They fail at tasks that a child would perform effortlessly. They hallucinate basic facts, lose track of simple contexts, or produce confident nonsense where even minimal understanding would suffice.

This tension is not new. It was articulated decades ago by the roboticist Hans Moravec, who observed that what is hardest for humans—abstract reasoning—can be relatively easy for machines, while what is effortless for humans—perception, motor coordination, common sense—remains extraordinarily difficult for AI. Today, this “Moravec’s Paradox” has become a feature of everyday life. A language model can help draft a philosophical argument or talk you through the quantum symmetrization postulate. Yet it fails to reliably count the number of objects in a simple image, or explain why you can’t push a ball through a wall.

This dissonance is not an illusion to be dispelled. It reflects something real: there is genuine intelligence at work, and yet it is not intelligence in the way we ourselves think. To understand how both of these can be true, we need a different perspective. Phenomenology—the philosophical investigation of experience and its structure—can show us why.

We can begin not with machines, but with our own experience. Consider something that seems entirely simple: perceiving a car. At first, perception unfolds as a continuous flow. As we move around the car, we encounter shifting appearances—shades of red, variations of smoothness, changing contours. There is no explicit thinking here, no deliberate judgment. There is simply the ongoing presentation of the object through changing perspectives. Yet this flow is not chaotic. It is structured. The different appearances cohere as appearances of the same thing. The car maintains its identity across variation.

Now suppose something interrupts this flow. We notice an abrasion on the surface. Our attention shifts. We no longer simply move through appearances; we pause and reflect upon them. We grasp the car as a whole in relation to its properties: the car is red, smooth, and damaged. This shift is decisive. What was previously a continuous perceptual synthesis has become articulated into a thing and its properties—a structure that can now be expressed in language. Where before there was simply the unfolding of appearances, there is now something that can be said. But language did not create this structure. It merely expresses it.

We do not first have words and then attach them to things. Rather, we are already oriented toward the world in a structured way—toward unities amidst change, toward things and their properties—and language gives expression to this orientation. Perception is not a stream of isolated stimuli. It is a coherent, anticipatory flow in which appearances are organized around enduring identities. The syntactic structures of language—subject and predicate, for instance—reflect this deeper organization within experience. And the syntactic articulation of language not only expresses this structure but can itself become an object of reflection, giving rise to the formalization of syntactic relations in logic.

___

Large language models are trained entirely on vast amounts of linguistic data—on the accumulated sediment of human expression. These systems do not perceive the world. They do not move through it.

___

But language does something else as well. It preserves meaning. Each time we say “this is red” or “that is smooth,” we leave behind a trace. Over time, these traces accumulate. Words and concepts become habitual—we can speak of a car being red or damaged without standing in front of one, without reactivating the perceptual experience from which these descriptions first arose.

Phenomenologists call this the sedimentation of meaning. As these syntactic structures of language are repeated and shared, they give rise to stable patterns of co-occurrence—regularities in how words appear together—that preserve, in sedimented form, the underlying structures of meaning from which they arise. Sedimentation is not merely a passive preservation of prior experience. Sedimented meanings press down onto experience itself, shaping how the world is anticipated and articulated in advance.

Sedimentation is both a strength and a danger. It allows meaning to stabilize, to be shared, to persist across time. But it also introduces the possibility of thoughtlessness—not as a rare failure, but as an everyday condition. Words can be used automatically, detached from the perceptual life that once gave them sense. We speak and judge without returning to the things themselves, repeating familiar formulations that circulate within our linguistic community. We say that “the economy is strong,” or that “markets are uncertain,” without any direct grasp of the concrete situations these phrases are meant to describe. What is said may be correct; it may even be well-formed. But it is no longer a genuine articulation of the world. It is not something we have seen for ourselves, but something we say because it is what “one” says.

Sedimented meaning, that is, the linguistic traces we leave behind—this is precisely the domain in which contemporary AI systems operate. Large language models are trained entirely on vast amounts of linguistic data—on the accumulated sediment of human expression. These systems do not perceive the world. They do not move through it. They do not form anticipatory horizons of experience. Instead, they operate on patterns that have already been articulated and stabilized in language. This is both the source of their power and the source of their limitation.

We can now return to Moravec’s Paradox. Why can AI perform sophisticated linguistic tasks yet fail at simple ones? Our answer is that what appears “simple” to us—perceiving a stable object, navigating a physical environment, recognizing what matters in a situation—is not simple at all. It rests on a vast, integrated structure of embodied experience. Language, by contrast, already contains a distilled form of this structure. It is the result of countless acts of perception and judgment, sedimented into communicable form.

AI systems are parasitic on this sediment. They inherit the products of intelligence without inheriting its living source. This is why they can be so genuinely impressive—they extend the reach of language, and with it, the reach of human intelligence. And it is why they remain so fundamentally limited—they lack the orientation toward the world that underlies understanding.

But there is something more that these systems lack—something that concerns not just how we experience the world, but how we answer to it. In human cognition, judgment is not merely the application of words. It involves an answerability to the world, a responsibility that characterizes our basic relationship to the world. Responsibility, here, is not an imported ethical appeal. The kind of answerability that is at stake is intrinsic to thinking and experiencing itself. Consider again the car. We approach it expecting a uniform red surface. But as we move around it, we discover a black, dented side. If this happens, we don’t simply update a data point. We stop and reflect. We return to the car itself, holding our previous judgment at arm’s length—treating it not as a settled description but as something that might have been wrong. We test it against what we now see. And if the judgment fails, we revise it—not because a rule tells us to, but because the world demands it.

This capacity for reflective self-correction is essential. It is what distinguishes genuine understanding from mere verbal fluency. When we judge, we put ourselves at risk. We commit to a claim that the world can expose as wrong, and we stand ready to answer for it.  Phenomenologists like Husserl, Heidegger, and Arendt worried deeply about this, because when we lose that answerability to lived experience, our thinking can drift into thoughtless repetition of public opinion in ways that are not just mistaken, but sometimes dangerous. 

AI systems do not possess the capacity for reflective self-correction. They produce language, but they cannot step back from it and hold it up against a world they experience. When a language model generates a confident falsehood—when it hallucinates—there is no moment of confrontation with reality, no tension between what was said and what is seen. The output simply continues, following the momentum of the plot. They extend meaning, but they cannot take responsibility for it.

SUGGESTED READING AI's growing threat to challenging ideas By Armin Schulz

Even when AI systems act, use tools, and update based on outcomes, the correction remains internal to a learned representational space. What is adjusted are trajectories within that space, guided by signals of success or failure. But this is not the same as relating those representations back to a world that is given as such. There is no standpoint from which the system can ask whether the representation itself is inadequate, or whether the situation calls for a return to the world—for renewed observation, for clarification of what is there.  That latter possibility is essential to human cognition. It is what defines the reflective stance and the constitution of a responsible speaker. It is in this movement that natural language itself is formed and refined.

It is commonplace to describe these systems as “next-word predictors.” But this description is misleading. It suggests a purely local, mechanical process of no significance—as though each word were chosen in isolation. A more accurate characterization is that they are plot extenders. They do not merely predict the next word; they narrate structured trajectories of sedimented meaning already set in motion by prior context. LLMs extend these plots purely from within—by following their internal momentum without access to the world they are about.

Seen in this light, AI is not an anomaly. It belongs to a long history in which fundamental structures of cognition are formalized and set into motion outside the organism. Writing formalizes the persistence of meaning—it allows thought to endure beyond the living speaker. Mathematics formalizes the structure of spatial and temporal relations, extending our perceptual grasp into domains beyond the reach of the senses. Computers formalize logical operations—the rule-governed inferences intrinsic to thinking—allowing thought to operate beyond the living subject. In each case, a capacity already at work in cognitive experience is given a new, independent form.

AI continues this trajectory, but what it formalizes is something different: the relational structure of meaning itself. Large language models represent meaning as positions and directions in high-dimensional mathematical spaces. Words that function similarly across vast histories of human expression end up near each other; trajectories of sense become geometrical paths. This is not a superficial trick. It is a formalization of the very structure that sedimentation produces—the web of relations between meanings that has accumulated over centuries of human thought and expression. It is why “plot extender” is not merely a metaphor. The trajectories these systems follow are mathematically real.

But these wonderful achievements—writing, mathematics, computation, AI—also generate products that take on a life of their own: texts, equations, models, outputs that circulate independently of the processes that produced them. And here lies a persistent temptation: to treat these products as more real, more authoritative, than the living activity from which they arise. To kick away the ladder of their source in lived experience and mistake the sediment for the source.

This temptation is especially acute with AI, because its products so closely resemble the outward expression of thinking. A well-constructed paragraph, a persuasive argument, a cogent analysis—these have always been signs of intelligence. When a machine produces them, the inference is natural: something must be thinking. But what phenomenology reveals is that thinking is not constituted by its products. It is constituted by an orientation toward the world—by the capacity to perceive, to judge, to be answerable for one’s claims. The product can be detached from this orientation. The thinking cannot.

What is at stake, then, is not how intelligent AI becomes. It is how we interpret what it produces. AI extends intelligence, bringing benefits for humanity, only insofar as we remain responsible for its interpretation—only insofar as we bring to its outputs the very capacities it lacks: perception, judgment, answerability to the world. It diminishes us not when it grows more capable, but when we forget that its meaning depends on ours.

AI is not artificial in the sense of being alien to us. It is made from the sediment of our own intelligence—formalized, extended, set into motion. But neither is it simply another mind among minds. It is a product, powerful and revealing, of the very capacities that define us. To orient ourselves to AI is not to decide whether it is intelligent. It is to recognize that we remain the source of its meaning, and that this responsibility cannot be delegated.