Short answer: training a language model to predict the next token can lead it to encode patterns about space, time, language and situations that were never written as explicit rules. Those representations can support useful behavior, but they are not proof that the model understands the world, has consciousness or reasons like a person. What a model has learned depends on its architecture, training stages, prompts, evaluation task and access to external tools.
What does “predict the next word” actually train?
Most large language models (LLMs) begin with a simple objective: given a sequence of tokens, estimate which token should come next. A token may be a word, part of a word, punctuation or another text fragment. During pretraining, the model adjusts billions of parameters to reduce prediction error across its training data.
The objective does not tell the model to build a map, learn physics or follow a chain of reasoning. However, text contains regularities that make those capabilities useful for prediction. Descriptions of locations contain spatial relationships; timelines contain temporal order; explanations contain causes, qualifications and procedures. A system that captures those regularities can predict text more accurately than one that merely memorizes nearby phrases.
This is an important distinction: the training goal is local prediction, while the internal machinery that helps achieve it can contain broader, reusable structure.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware match#1 Best Overall
What evidence shows up inside a model?
Spatial and temporal representations
Wes Gurnee and Max Tegmark’s ICLR 2024 paper, Language Models Represent Space and Time, examined internal activations in studied language-model families, including Llama-2. Their analyses found representations that track aspects of location and time. In practical terms, parts of the network responded in systematic ways to where something is and when an event occurs.
That result is evidence of encoded structure, not a demonstration of a complete “world model.” The authors describe spatial and temporal information as basic ingredients that could contribute to one. A dynamic causal model would need to represent how situations change, what causes those changes and how interventions alter outcomes. The activation analyses alone do not establish all of that.
Representation is not human understanding
An activation can carry information without the system having an inner experience of that information. The finding does not show consciousness, beliefs, intentions or a human-like mental model. Nor does it guarantee that the representation will be used correctly on a new prompt.
Performance and representation should therefore be reported narrowly: identify the model family, the task, the probing or analysis method and the conditions under which the behavior appeared.
Rank #2
Why capabilities can look as if they “emerge”
Some abilities appear weak or absent in smaller models and become visible in larger ones. Jason Wei and colleagues used “emergent abilities” for capabilities that meet that pattern under their definition in a 2022 paper. A sharp change in measured accuracy can make a capability look as though it suddenly appeared once a model crossed a scale threshold.
That interpretation is contested. Sheng Lu and colleagues argued in a 2023 paper that some reported examples can result from a combination of in-context learning, memorized information and linguistic knowledge. Evaluation choices can also create an apparent jump: a metric may be insensitive to partial progress in smaller models, then register a large improvement once answers pass a scoring threshold.
| Question | Emergence interpretation | Alternative interpretation |
|---|---|---|
| What is observed? | A capability is not detected in smaller models but is detected in larger ones. | The underlying improvement may be gradual, while the test or scoring rule makes it look sudden. |
| What may explain it? | Additional scale enables a qualitatively new competence. | In-context learning, stored knowledge and language skill combine to solve the task. |
| What must be checked? | Whether the pattern holds across model families, prompts and metrics. | Whether alternative metrics reveal continuous improvement and whether memorization or prompting accounts for the result. |
Neither account should be treated as a universal law. “Emergent” is a description of an observed scaling pattern plus an interpretation of its cause, not proof that a hidden faculty switched on.
What the model may be learning beyond surface word matching
Language structure
Models capture syntax, word senses, discourse patterns and relationships between phrases. This lets them complete a sentence in a grammatically plausible way and adapt to a style or format shown in the prompt. It does not mean they possess a dictionary-like, human representation of every concept.
Regularities about places and events
Text associates objects with locations and events with dates or sequences. Internal spatial and temporal signals can help a model answer questions or generate descriptions, but the signals may be incomplete, inconsistent or tied to the wording of the examples that produced them.
Procedures and problem-solving patterns
Explanations, worked examples and code expose recurring steps. A model can reproduce those patterns and sometimes combine them in unfamiliar situations. Whether this is reliable reasoning depends on the task, context and verification method. A fluent chain of steps can still contain an incorrect premise or an invented result.
Multilingual relationships—with uneven coverage
Shared parameters let a model connect patterns across languages, especially where training data provides translations or parallel concepts. Coverage is not equal, however. A model may perform strongly in English and much less consistently in languages with less representation, different writing systems or fewer high-quality examples. A capability demonstrated in one language should not automatically be generalized to all others.
Pretraining, later training and tools are different sources of capability
It is easy to attribute every impressive response to what the model learned during next-token pretraining, but a deployed system may have several additional components.
- Pretraining: exposure to large text collections while optimizing token prediction.
- Later training: supervised examples, preference optimization or other procedures that shape helpfulness, refusal behavior and instruction following.
- Prompting and in-context learning: information and examples supplied at the time of a request, without changing the model’s stored parameters.
- External tools: retrieval systems, calculators, code execution, browsers or other software that provide information or perform actions during inference.
The BetaNews article that prompted this question discusses retrieval, reflection and tool use as features of some AI systems. Those features should not be presented as capabilities that every LLM acquires from basic pretraining. When evaluating an example, ask which component produced the result.
Why fluent output can still fail
Confidently wrong answers
Next-token optimization rewards likely continuations, not truth. If the training distribution contains contradictions, gaps or persuasive errors, a model can produce a plausible but false answer. Fluency is evidence that the output fits learned patterns; it is not independent verification.
Brittleness under small changes
A minor wording change, an unfamiliar format or a request that requires several dependent steps can expose gaps in the learned representation. High performance on one benchmark prompt may not transfer to a paraphrase or a real-world situation.
Memorization and generalization are hard to separate
A response may reflect a remembered passage, a recombination of familiar language or a genuinely useful abstraction. Tests need controls for training-data overlap and prompts that distinguish recall from applying a relationship to a new case.
Recommended Free Tools
Best Value
Uneven world knowledge
Information changes after training, appears unevenly across languages and may be missing from the data entirely. A model without retrieval or another current source cannot be assumed to know a recent event simply because it can discuss the topic fluently.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to evaluate a claim about what an AI “knows”
- Define the claim precisely. Is it about an internal representation, task accuracy, factual recall, planning, tool use or something else?
- Name the system. Record the model family and version, and whether it was pretrained only or received later training.
- Describe the conditions. Include the prompt, examples in context, language, temperature or other relevant settings, and any retrieval or software tools.
- Use more than one test. Check paraphrases, new examples and adversarial cases rather than relying on a single demonstration.
- Test alternatives. Look for memorization, in-context learning, linguistic shortcuts and metric artifacts that could explain the result.
- Separate encoding from reliability. A probe may show that information is present in activations, while task tests show whether the model can use it consistently.
So what was the AI taught?
It was taught an optimization rule and exposed to data—not an explicit lesson titled “understand space,” “reason about time” or “use a tool.” To reduce prediction error, the model can develop internal features that encode relationships useful for many texts. Those features may support behavior that looks like knowledge or reasoning.
The strongest defensible conclusion is limited: learned representations and task abilities can go beyond literal phrase copying, and their origins are often shared among scale, data, prompting, later training and tools. Current evidence does not justify calling those representations consciousness, a complete causal world model or dependable human-like understanding.
Quick Recap
Further reading
- Wes Gurnee and Max Tegmark, Language Models Represent Space and Time, ICLR 2024.
- Jason Wei and colleagues, Emergent Abilities of Large Language Models, 2022.
- Sheng Lu and colleagues, Are Emergent Abilities in Large Language Models just In-Context Learning?, 2023.
- Keryn Gold, “Beyond words: What AI is really learning—and what it knows that we never taught it,” BetaNews, April 12, 2025.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Free tools Windows power users keep installed
One-click scans. No signup required.
