SuperpositionA useful representation can capture relationships that would otherwise require treating every detail separately.

What It Is

An image with a million pixels contains a million values, but a recognizable face does not use them arbitrarily. Eyes appear above the nose, and the nose above the mouth. In a landscape, a horizon separates sky from ground. These relationships constrain the combinations that can occur.

A model that captures those relationships can represent the image with fewer independent quantities. Superposition describes how complex, high-dimensional information can be encoded in fewer dimensions by using its intrinsic structure. The apparent complexity already occupies a smaller, structured space.

This is why a useful mental model can describe something that initially seems too complex to understand. It identifies the relationships that organize the variation, rather than simply discarding inconvenient details.

The article's compression argument relies on structure in the material; merely omitting details is a different operation.

Read this diagram

Observe many details → Find shared structure → Represent that structure → Check what it preserves.

The Core Principle

A containing space can have many dimensions while the data uses only a small fraction of its possibilities. Correlations and recurring patterns constrain the data to a lower-dimensional structure.

Phenomena that appear complex often have a far smaller intrinsic structure.

The smaller representation works because it identifies the structure the data actually has. Arbitrarily choosing fewer variables would not provide the same result.

The Manifold Hypothesis

The manifold hypothesis, from machine learning, proposes that high-dimensional data usually lies on or near a much lower-dimensional manifold embedded in the larger space. Much of the apparent complexity therefore comes from the size of the containing space, rather than the variation the data actually exhibits.

Observable Examples

Natural images occupy a million-dimensional pixel space, but faces and landscapes share structural relationships. Eyes above nose above mouth, or sky separated from ground by a horizon, rule out most arbitrary arrangements. Actual images occupy only a small fraction of the possible pixel combinations.

A 10,000-word vocabulary also permits far more combinations than meaningful conversation uses. Grammar, semantics and context constrain how words can combine, so actual conversation occupies a smaller space.

Daily behavior has a similar pattern. Infinitely many action sequences are possible, but a handful of default scripts account for what usually happens: a morning routine, a work sequence and an evening wind-down, for example. For images the smaller structure is measured across large collections of examples; for one person's days it is whatever that person's own observation turns up.

Why Simple Models Work

The simple-model argument follows from this structure. A good model identifies the dimensions where the important variation occurs. When a few factors explain most of it, the remaining dimensions contribute noise rather than signal.

The model is not merely an inaccurate approximation tolerated because it is easy to teach. It has found the actual structure that makes the phenomenon predictable.

Practical Implications Table

These models are useful when they identify the low-dimensional structure of the behavior itself. The apparent number of possible causes does not determine how many variables the working model needs.

Superposition in Information Encoding

Superposition also describes multiple features sharing the same representation space. Instead of giving each feature a separate slot, a distributed representation spreads features across dimensions as weighted combinations.

The features overlap, but pattern separation can recover them. That allows a limited representation space to hold more information than dedicated slots would.

Working Memory Connection

One account of working memory applies the same idea: its items, approximately 4 to 7 at once, are overlapping patterns of activation rather than separate storage slots. Distributing the representation allows greater information density, but the capacity remains limited.

When the complexity exceeds what working memory can hold, externalize it. Trying to keep more in mind does not remove the limit on the distributed representation.

Compression Through Structure Discovery

Compression works when data already contains relationships that can be represented more economically. The difference is between discarding dimensions arbitrarily and discovering which dimensions carry the variation:

Application: Tracking

Trying to track everything is impossible. The practical aim is to find the few variables that explain most of the variation.

Suppose someone tracks 30 variables over 30 days and finds that sleep quality moves closely with exercise and with screen time and not at all with 20 of the other factors. Reducing the record to the relevant 2-3 variables loses little predictive power and cuts the tracking work by about 90%, since 27 or 28 of the 30 variables are dropped.

The reduction follows the observed relationships. It discovers which dimensions matter for the system instead of deciding in advance which details to ignore.

Framework Integration: How Superposition Appears

The mechanistic frameworks use smaller representations of apparently unbounded behavior:

Each framework aims to identify existing structure. Its value depends on that structure explaining the result, rather than on smallness alone.

Pedagogical Magnification and Resolution Matching

Pedagogical magnification matches the resolution of analysis to the phenomenon's intrinsic dimensions. Examining 100 variables when five carry the variation spreads a limited compute budget across irrelevant detail. Focusing on the dimensions that actually vary allows deeper analysis of what matters.

A database decision illustrates the difference:

The decision has a few relevant dimensions for the use case. Treating it as a 50-dimensional problem consumes attention without improving the decision.

Discretization as Compression

Discretization identifies the natural divisions in a process. "Work on project for 3 hours" leaves an uncountable sequence of moments to manage. "Complete 6 25-minute blocks" gives six units to complete.

The blocks work because concentration tends to come in periods separated by rest. The discrete representation follows that structure instead of treating every moment as independent.

Observable Questions and Dimensionality

An unbounded question such as "How can I be better?" can search the whole knowledge graph. It leaves infinitely many dimensions open and may never finish, or return an arbitrary answer.

Question Theory constrains that search. "What's one improvement to work launch sequence?" specifies a system and leaves a handful of possible improvements to consider. The narrower space supports a search that finishes with an actionable answer.

Practical Applications

Application 1: Problem Simplification

Begin by observing the problem broadly. If it appears to involve 50 variables, track it for 30 days and inspect which variables actually change with the result. When a few factors explain most of the variation, focus on those and set aside the decorrelated noise.

Productivity may appear to depend on more than 20 variables, including sleep, diet, exercise, environment, tools, motivation, mood, weather and social interactions. After 30 days of tracking, the example finds that most of the variation is explained by three factors: consistent wake time, completion of a morning braindump, and available morning resources. Those three then become the targets for improvement.

Application 2: Framework Selection

A useful framework explains much of the variation with few enough variables to use:

The expected-value model is more useful in this comparison despite lower theoretical accuracy. Four variables fit within working memory, each can be manipulated, and together they retain the essential structure. A model that cannot be computed in the available capacity is harder to act on.

Application 3: Mental Model Evaluation

Ask whether the model reduces complexity while preserving predictive power. "Procrastination = activation energy exceeds available willpower" uses two variables, accounts for many instances, and suggests interventions on either side of the comparison.

"Procrastination = psychological resistance" names the difficulty without specifying variables. It explains no particular outcome and gives no way to intervene.

This connects Occam's razor to compression: the simplest explanation that fits the data is the best candidate for the intrinsic structure that matters for prediction.

The Tracking Optimization Protocol

The goal is the smallest set of measurements that preserves the most explanatory power:

1. Initial phase (Days 1-30):
   - Track 10-15 variables (inputs + outputs)
   - Include everything potentially relevant

2. Analysis phase (Day 31):
   - Calculate correlations between all variables
   - Identify which inputs predict which outputs
   - Find clusters of correlated variables

3. Compression phase:
   - Keep 1 representative from each cluster
   - Drop uncorrelated variables (noise)
   - Result: a few key variables capturing most of the variance

4. Validation phase (Days 32-60):
   - Track only compressed set
   - Verify predictive power maintained
   - Adjust if needed

5. Maintenance:
   - Continue with minimal set
   - Periodically check if intrinsic structure changed

The validation phase matters because the structure has to be discovered for this person and checked against later observations. The protocol does not assume that everyone has the same relevant variables.

Integration with Mechanistic Frameworks

Connection to Working Memory

The small capacity of working memory constrains how many dimensions can be processed at once. A problem with many intrinsic dimensions is more than it can hold.

There are three ways to handle the gap: externalize the work in a journal or on a whiteboard, find a smaller structure, or process subsets sequentially. Externalization lets the representation extend beyond biological working-memory capacity.

Connection to State Machines

State machines represent behavior as discrete states and transitions. Infinitely many next moments are possible, but a handful of default scripts organize actual behavior. The state machine is useful because those routines already supply a small structure to represent.

Connection to Expected Value

Motivation can appear to depend on mood, circumstances, personality, energy, time of day and recent events. The expected-value account reduces these to four variables: motivation rises with the reward and with the chance of getting it, and falls with the effort required and with the time until the reward arrives. A drop in motivation is then a change in one of the four.

Where the account works, its value comes from having found that structure, rather than from being a convenient approximation.

Connection to Question Theory

Question Theory relates the cost of a search to its dimensionality. "What should I do?" leaves the search unbounded, described as O(∞). "What's next action on highest-priority task?" restricts it to one dimension, described as O(1).

A well-specified question narrows the search to the space where the relevant answer can be found.

Common Misunderstandings

Misunderstanding 1: "Simple = Simplified"

A simple model can be accurate because the phenomenon itself has few independent dimensions. When behavior has a three-variable structure, a three-variable model represents that structure without needing to be a dumbed-down account of something more complex.

Misunderstanding 2: "More Variables = More Accurate"

If five dimensions carry the relevant variation, recording 50 can make the model worse. The additional variables overflow working memory, introduce random correlations, obscure the pattern and reduce accuracy through overfitting.

Find the five that matter and examine them closely.

Misunderstanding 3: "Context Always Matters"

Much of the apparent contextual complexity can turn out to be decorrelated noise. If most contextual factors have zero correlation with the outcome, they can be ignored for that decision. The relevant task is to distinguish signal from noise, rather than include every available detail.

Observable Patterns

Pattern 1: The 80/20 Distribution

A familiar rule of thumb holds that about 20% of variables explain 80% of the variation. The tracking examples include:

• Exercise and screen time explain most of the variation in sleep quality.
• Wake-time consistency, a braindump and environment explain most of the variation in work output.
• Sleep, exercise and social factors explain much of the variation in mood.

Where the distribution holds, it follows from lower-dimensional structure in the thing being tracked. It does not require a separate universal law.

Pattern 2: Compression Resistance Reveals Noise

Signal compresses when a few variables account for most of its variation. Noise requires many variables that each contribute little.

If reaching 90% accuracy requires 30 variables that each explain 3%, you are probably fitting noise instead of identifying the intrinsic structure.

Pattern 3: Framework Convergence

Several frameworks arrive at similarly small representations. Willpower has a few main depletion sources. Activation energy has a few main threshold factors. Expected value uses four variables, and state machines use a handful of typical states.

The agreement is a reason to begin a new question by looking for a handful of variables.

Anti-Patterns

Anti-Pattern 1: Premature Compression

Assuming "calories in vs calories out" before observing the system decides its dimensionality too early. A one-dimensional account might fit, or the actual structure might require three variables. Track the system first, then reduce the representation based on what you find.

Anti-Pattern 2: Forcing Arbitrary Dimensions

Choosing measurements solely because a theory recommends them can miss the relevant structure. Tracking macronutrients because "nutrition science says so" may not identify the variables that matter for this N=1 case. Begin broadly and let the observed correlations guide the selection.

Anti-Pattern 3: Ignoring Manifold Shifts

A three-variable model that worked for six months may stop working after a new job, a move or a relationship change. The relevant structure can change with the system. Periodically check that the model still fits current data.

Related Concepts

• Pedagogical Magnification matches analysis to intrinsic dimensionality.
• Working Memory limits how many dimensions can be handled at once.
• Discretization finds useful divisions in a process.
• State Machines represents behavior through discrete states.
• Tracking discovers which variables explain variation.
• Expected Value uses four variables to describe motivation.
• Question Theory bounds the space searched for an answer.
• Computation as Core Language develops the information-theory foundation.