Oct. 17, 2026

People are seriously discussing AI consciousness and describing chatbots as though they were living entities.

(For simplicity, let’s say that any LLM, like Claude or ChatGPT, is what we mean by “AI.”)

I think this is a category error, at least with current technology. Here’s one more way to think about why…

I recently came across this quote from Amy Hoy:

llms only “exist” to give one reply at a turn

you: x
llm: y

then the whole process starts over. there is no continuity. each reply is 100% fresh. the context of your previous “conversation” is included, hidden from you, in every request you make. continuity is an illusion.

Source: Amy Hoy

This is exactly what is happening. The harness (i.e., infrastructure surrounding the model) is what creates the illusion of continuity.

In fact, it’s even more segmented than the above quote suggests: every single token (i.e., word or part of a word) is the product of an individual calculation.

Even if you think this is analogous to how human brains work, I’d argue that the absence of processing between turns is fundamentally different. In my view, continuous experience moment-to-moment is a core aspect of what people ordinarily mean by the term consciousness.

Humans generally have continuous experience, except with limited interruptions (e.g., under anesthesia). AI is the opposite: it has no continuous experience—just individual calculations stitched together following user prompts.

If I’m wrong, then what exactly do LLMs contemplate during the limbo between user prompts?