How LLMs Learned to Think

The short answer: They didn’t. LLMs don't think. What appears as reasoning is the resolution of constraints in structured fields.

Why Are We so Mystified by How LLMs Work?

Much of the published research on LLMs, including articles from established journals like Computational Linguistics, fails to provide structural models of how language models work. Instead, it relies on observations (LLMs sometimes “hallucinate”), normative prescriptions (“we need better training data”), or speculative hopes (“diversity might improve outcomes”).

But none of these statements rest on an articulated causal theory. Why should better data help, if the architecture cannot reason? What exactly counts as “reasoning,” and where does it emerge from?

Instead of symbolic logic, I argue LLMs operate through constraint resolution across token fields — a defining characteristic of field-based intelligence. This concept underpins the model's ability to appear to reason, despite lacking internal world models or goals.

The inference in LLMs is constraint collapse in a high-dimensional semantic field.

This is not speculative. It is the closest descriptive analogue to what transformer math implements:

• Embeddings = token states in a semantic field

• Attention = weighted constraints reshaping local fields

• Layer stacks = multi-resolution refinement of coherence

• Output = coherent vector collapse to token space

• And the instructions on how transformers calculate are structurally encoded in language and LLM learn the pattern of this structure and encode it in their weights. Language isn’t just symbolic — it encodes procedures: logic, recursion, conditionals, and constraints. These mirror computational structure.

When we say: “If X, then Y,” or “Given A, assume B,” we’re expressing constraint propagation. LLMs trained on trillions of tokens see this structure repeatedly, without needing to be told that it is logic. Transformers don’t understand meaning; they statistically absorb how humans resolve ambiguity, sequence information, or apply constraints through language. This structure is encoded in their weights.
LLMs are not “doing reasoning” as a separate task. Their architecture is a scaffold for absorbing and reenacting structured symbolic behavior — because language is already a compressed trace of cognition.

The Natural Language Processing (NLP) field continues to propose solutions without defining the problem. If the issue is reliability, it is a question of statistical behavior, not bias. If the issue is hallucination, then one must ask: why are we surprised that a system with no grounding invents plausible-sounding errors?

Too Often, the Discourse Treats Symbolic Errors as Moral Ones

A machine that produces biased output is not biased — it is ungrounded. The anthropomorphic language (“racist model,” “lazy answer,” “poor reasoning”) imports human categories without mechanistic clarity. If there is no internal model of truth, no feedback loop, and no verification process, then there can be no error in the epistemic sense. Only divergence from human expectation.

Thus, current critiques of LLMs too often misidentify performance variance as ethical failure, and prescribe solutions without constraint analysis. A disclaimer does not constitute a fix. And the suggestion that we need to “improve the data” or “enhance transparency” remains meaningless without a theory of symbol-field alignment.

Until we define what reasoning is structurally — and what its absence means — we will mistake fluency for cognition and correction for learning.

My previous articles The Theory of Between and Field-Based Intelligence already explained that human cognition and language model dynamics are structurally isomorphic processes governed by constraint resolution within a tensor field. Token prediction can be accurately modelled as a process of "coherence collapse," wherein possible interpretations converge to a singular, contextually coherent outcome. A symbol gains meaning by where and when it appears. A given language has a finite number of symbols which are arranged sequentially to encode communication. This sequential ordering can be seen as travelling through a semantic field under constraints. Therefore, symbolic systems require rules that define sequences: where they begin, where they end, and how they proceed. We use symbols to encode concepts, often conflated with "words," but the correct unit is the minimum set of symbols required to initiate communication. Words shift in meaning with context, because language structurally encodes reasoning. LLMs approximate reasoning because language, used correctly, reflects structured human inference.

But reasoning is field-specific and therefore language-specific. Different doesn’t mean better or worse.

Meaning results from symbols traversing a constrained field. However, meaning does not travel—only symbols do. The capacity of the receiver to reconstruct meaning, even when symbols are partially incorrect or missing, depends on the symbolic choices made. All languages must define correct symbolic ordering, but deviations from those rules do not necessarily prevent reconstruction. They may even be intentional, and correctness cannot be judged solely by symbol reading. Meaning can be reconstructed despite partial loss of symbolic transmission, depending on the form of the symbol and provided the losses are evenly distributed proportionally to how symbol and meaning correlate—but not symbol alone. Mainstream theory in AI or linguistics cannot explain why large language models work because classical statistical learning only describes curve-fitting over data.

The Breakthrough of Transformers Was Not Just Speed or Scalability

Transformer implicitly show that meaning emerges from the geometry of token interactions. But this has not been formally realized.

This article complements the previous articles with an initial overview of how the transformer architecture and encoded reasoning in language together create the effect we observe interacting with LLM.

Key concepts:

Reasoning = Constraint collapse

Meaning = Vector alignment shaped by usage

Inference = Emergent coherence, not symbolic traversal

Limits = No grounding, no abstraction, no intent  

What Is a Vector?

LLMs create ultra-rich representations of words, sentences, or entire documents by encoding complex relationships across hundreds or thousands of dimensions, typically as a vector. For example, 12,288-dimensional vectors. Each dimension is a 32-bit floating-point number (FP32), allowing around 8.4 million distinct values per dimension. Hypothetically, this means the total number of representable vector combinations exceeds the number of atoms in the observable universe (for more background on vector encoding see appendix).

Humans reason hierarchically and symbolically. LLMs cluster probabilistically. A whale may encode 30% “submarine” and 70% “mammal” — not because it’s confused, but because language use reflects functional analogies, not definitions.

Language models thus operate in a shared concept space shaped by distributional patterns. Even across languages, embeddings often align by meaning — not by vocabulary.

So while linear arithmetic may be visible in early embeddings, actual inference involves:

• Constraint propagation across layers,

• Selective attention to tokens that modify meaning potentials,

• Incremental pruning of the prediction space.

Field-Based Intelligence, Not Arithmetic

Classical logic operates with fixed categories, explicit truth conditions, and stepwise deduction. Inputs deterministically lead to outputs via discrete rules. Reasoning in this mode is hierarchical, symbolic, and enumerable. Field-based intelligence, by contrast, is emergent, probabilistic, and relational. In transformer models, each token's meaning is dynamically shaped by its context through attention. There are no classical axioms, no fixed meanings—only constraint propagation until a coherent field collapses.

For instance, when prompted with “Berlin is to Germany as X is to France,” the model does not calculate a capital city using logic. Instead, it collapses the statistical field of co-occurrence patterns into the most coherent token based on prior data: “Paris.”

Field-based intelligence is not a fuzzy approximation of logic. It is a system’s ability to generate coherent outputs by resolving constraints within a dynamic representational field, rather than computing explicit symbolic steps. It is a structurally distinct form of resolution—coherence not through computation, but through constraint. This redefines inference: not as traversal through propositions, but as the collapse of a constrained semantic manifold.

A transformer works by converting input tokens into high-dimensional vectors through a linear transformation. These vectors are processed through layers that apply attention mechanisms and feed-forward networks. The final decoder output is passed through a softmax function to generate a probability distribution over possible next tokens.

Each token is not a word but a subword or character-level fragment. The model learns to map these to embeddings, and its attention mechanism determines which parts of the input context are most relevant. Multi-head attention means the model applies multiple attention patterns simultaneously, each capturing a different type of relationship.

Meaning Emerges from Constraint Resolution

As the token progresses through layers, its vector is dynamically reshaped — influenced by all other tokens in scope. Meaning does not reside in any one token; it is emergent from movement through the network. The model does not “look up” a meaning but constructs a field that collapses to a high-probability outcome, shaped by weights, position, and context.

Transformer-based models do not perform logic or arithmetic on stable representations. They collapse constraint fields — shaped by prior exposure to language — into coherent outputs. Each token is not computed; it is selected through the narrowing of a probabilistic field shaped by attention and prior tokens.

LLM completions are computed in one pass, not incrementally like human deliberation. All vectors propagate forward at once through the transformer. There is no stepwise logic, pause, or update cycle. The model does not predict a word, wait, and continue. It collapses a semantic field into a single coherent output based on all context in a single forward pass.

The transformer architecture does not use “neurons” in the classical sense. What matters is not neuron-like behavior but linear transformations followed by nonlinearities applied to vectors — not discrete neurons with functional roles. Referring to these structures as neurons misleads: there is no localized symbolic unit with semantic meaning, only distributed matrix operations that adjust the entire token embedding.

In LLMs, meaning is not stored or retrieved. It is emergent from interaction. The representation of a word is always situated: it exists only in relation to others in the current prompt and in alignment with layer-wise transformations. Attention weights modulate the influence of other tokens; feed-forward layers deform the latent field. Nothing is “understood”; everything is re-aligned.

There is no mental model of the user, the world, or the sentence — only weighted patterns of association. That these patterns often simulate belief, intention, or logic is a function of training on human language, not of internal competence.

Limits of current LLM Architecture

LLMs are not linguistically neutral. The language of their training impacts their 'reasoning' capacities because tokens representing English or German activate different vector patterns. Each language encodes different grammatical constraints and semantic assumptions. Reasoning, therefore, is not universal — it is language-specific. Moreover, model drift presents a structural issue. Human language usage is subtly shifting over time, but LLMs are frozen snapshots of linguistic distributions at a particular training point. Because the models do not understand meaning but only approximate usage patterns, any drift introduces a cumulative error. If training data is not continually aligned with linguistic evolution, the model’s output will diverge further from contemporary coherence. This is not just a temporal lag. It is structural stasis: the model's 'intelligence' is bound to the availability of new, high-quality text. As linguistic patterns shift, a model that cannot update its field alignment will accrue epistemic error. Intelligence in LLMs trails the cultural evolution of language itself.

Transformers are powerful statistical approximators, but they are bounded by design:

• No grounding: Tokens are unmoored from sensory or causal reference. They float in wordspace, not worldspace.

• No abstraction: The architecture cannot bind variables, generalize from form, or perform recursive operations without external scaffolding.

• No error correction: Transformers do not revise internal models; they only adjust probabilities layer by layer.

• No intent: Outputs are not selected for truth, coherence, or ethics — only for contextual likelihood.

• No dynamic learning: they have no persistent memory and no capacity to update their weights dynamically. They do not learn from use or interaction.

Thus, when a model appears to demonstrate "theory of mind," or analogical reasoning, it is doing neither. It is reproducing statistical shadows of texts in which humans performed those acts. Current models have likely ingested most high-density linguistic material produced by humans over the past 400 years. This helps explain the performance plateau observed in recent models despite massive increases in parameter counts. Once the statistical structure of high-quality language has been learned, adding more of the same yields diminishing returns. It may also explain why models like DeepSeek perform relatively well in Chinese: the model has likely been trained on a richer and more representative Chinese corpus than earlier multilingual models. However, a direct 1:1 performance comparison between languages is misleading — not because one model is “better,” but because each language encodes reasoning differently. Reasoning in language models is shaped by the distributional density and grammatical structure of the training corpus, not by any universal logic engine. In this sense, performance differences across languages reflect differences in symbol-field alignment, not intelligence. Scaling alone will not yield new capabilities. They must be trained on language that encodes thinking as process, not performance. That requires new kinds of writing — not just synthetic data, but structurally explicit human logic expressed in text. Until then, superhuman intelligence is highly unlikely.

Appendix

A vector looks like:

[0.12, -0.98, 0.34, 0.76, -0.45, ..., 0.05]

encodes multiple, entangled attributes rather than discrete features. A value near 1 suggests strong association with an abstract feature; a value near 0 or negative suggests weak or inverse correlation.

For example, an embedding for "cat" might include:

• 0.82 → Furry animals

• -0.12 → Reptiles (low value)

• 0.30 → Household pets

• 0.95 → Mammals

• 0.25 → Independence

If “tiger” activates overlapping dimensions similarly, the model clusters them. Embeddings are learned by observing how words are used, not what they are. The system clusters by usage, not essence. This is why “tree” and “book” may end up close in vector space — books are made from trees — even though they do not share a category in any human taxonomy.

There is no stable vector for any token such as "Berlin" or "France"; there is only the field-state induced by current context and past token activations. When the model is prompted with an analogy like “Berlin is to Germany as X is to France,” it does not solve an equation. Instead, it collapses the semantic field shaped by that structure into a likely continuation. The model identifies that such prompts typically resolve with capital cities and that “Paris” is a frequently co-occurring token in this analogy frame. It does not know what Paris or Berlin are — it only encodes that their directional relationships to Germany and France, respectively, are statistically similar. “Paris” is more likely than “Tokyo,” but “Tokyo” is still possible: it lies within a lower-probability region of the same semantic field. Resolution is not about meaning but about prior structure — what has occurred, not what is true.

A model’s output is influenced by temperature: a sampling parameter that controls whether the model always selects the most likely token or allows for less likely alternatives. A temperature of 0 forces deterministic output; higher values inject stochasticity. This makes outputs varied and context-sensitive — not error-prone, but consistent with field resolution under uncertainty.

A pdf version is available here.