Superposition

What It Is

An image with a million pixels contains a million values, but a recognizable face does not use them arbitrarily. Eyes appear above the nose, and the nose above the mouth. In a landscape, a horizon separates sky from ground. These relationships constrain the combinations that can occur.

A model that captures those relationships can represent the image with fewer independent quantities. Superposition describes how complex, high-dimensional information can be encoded in fewer dimensions by using its intrinsic structure. The apparent complexity already occupies a smaller, structured space.

This is why a useful mental model can describe something that initially seems too complex to understand. It identifies the relationships that organize the variation, rather than simply discarding inconvenient details.

The article's compression argument relies on structure in the material; merely omitting details is a different operation.

Read this diagram

Observe many details → Find shared structure → Represent that structure → Check what it preserves.

The Core Principle

A containing space can have many dimensions while the data uses only a small fraction of its possibilities. Correlations and recurring patterns constrain the data to a lower-dimensional structure.

Most phenomena have roughly 3-10 dimensions of intrinsic structure, despite appearing much more complex.

The smaller representation works because it identifies the structure the data actually has. Arbitrarily choosing fewer variables would not provide the same result.

The Manifold Hypothesis

The manifold hypothesis proposes that high-dimensional data usually lies on or near a much lower-dimensional manifold embedded in the larger space. Much of the apparent complexity therefore comes from the size of the containing space, rather than the variation the data actually exhibits.

Observable Examples

Natural images occupy a million-dimensional pixel space, but faces and landscapes share structural relationships. Eyes above nose above mouth, or sky separated from ground by a horizon, rule out most arbitrary arrangements. Actual images occupy only a small fraction of the possible pixel combinations.

A 10,000-word vocabulary also permits far more combinations than meaningful conversation uses. Grammar, semantics and context constrain how words can combine, so actual conversation occupies a smaller space.

Daily behavior has a similar pattern. Infinitely many action sequences are possible, but roughly 5-10 default scripts account for what usually happens: a morning routine, a work sequence and an evening wind-down, for example.

Why Simple Models Work

The simple-model argument follows from this structure. A good model identifies the dimensions where the important variation occurs. When 3-5 factors explain most of it, the remaining dimensions contribute noise rather than signal.

The model is not merely an inaccurate approximation tolerated because it is easy to teach. It has found the actual structure that makes the phenomenon predictable.

Practical Implications Table

These models are useful because they identify the low-dimensional structure of the behavior itself. The apparent number of possible causes does not determine how many variables the working model needs.

Superposition in Information Encoding

Superposition also describes multiple features sharing the same representation space. Instead of giving each feature a separate slot, a distributed representation spreads features across dimensions as weighted combinations.

The features overlap, but pattern separation can recover them. That allows a limited representation space to hold more information than dedicated slots would.

Working Memory Connection

Working memory uses distributed encoding. Its 7±2 items are patterns of activation across neural tissue, rather than seven separate storage slots. Distributing the representation allows greater information density, but the biological capacity remains limited.

When the complexity exceeds the 4-7 items working memory can hold, externalize it. Trying to keep more in mind does not remove the limit on the distributed representation.

Compression Through Structure Discovery

Compression works when data already contains relationships that can be represented more economically. The difference is between discarding dimensions arbitrarily and discovering which dimensions carry the variation:

Application: Tracking

Trying to track everything is impossible. The practical aim is to find the 3-5 variables that explain most of the variation.

A tracking example begins with 30 variables over 30 days. Sleep quality correlates 0.87 with exercise, 0.72 with screen time and roughly zero with 20 other factors. Reducing the record to the relevant 2-3 variables loses less than 10% of predictive power while cutting tracking work by 90%.

The reduction follows the observed relationships. It discovers which dimensions matter for the system instead of deciding in advance which details to ignore.

Framework Integration: How Superposition Appears

The mechanistic frameworks use smaller representations of apparently unbounded behavior:

Each framework identifies existing structure. Its value depends on that structure explaining the result, rather than on smallness alone.

Pedagogical Magnification and Resolution Matching

Pedagogical magnification matches the resolution of analysis to the phenomenon's intrinsic dimensions. Examining 100 variables when five carry the variation spreads a limited compute budget across irrelevant detail. Focusing on the dimensions that actually vary allows deeper analysis of what matters.

A database decision illustrates the difference:

The decision has roughly 3-5 relevant dimensions for the use case. Treating it as a 50-dimensional problem consumes attention without improving the decision.

Discretization as Compression

Discretization identifies the natural divisions in a process. "Work on project for 3 hours" leaves an uncountable sequence of moments to manage. "Complete 6 25-minute blocks" gives six units to complete.

The blocks work because concentration naturally occurs in periods separated by rest. The discrete representation follows that structure instead of treating every moment as independent.

Observable Questions and Dimensionality

An unbounded question such as "How can I be better?" can search the whole knowledge graph. It leaves infinitely many dimensions open and may never finish, or return an arbitrary answer.

Question Theory constrains that search. "What's one improvement to work launch sequence?" specifies a system and leaves roughly 5-10 possible improvements to consider. The narrower space supports a search that finishes with an actionable answer.

Practical Applications

Application 1: Problem Simplification

Begin by observing the problem broadly. If it appears to involve 50 variables, track it for 30 days and inspect which variables actually change with the result. When 3-5 factors explain most of the variation, focus on those and set aside the decorrelated noise.

Productivity may appear to depend on more than 20 variables, including sleep, diet, exercise, environment, tools, motivation, mood, weather and social interactions. After 30 days of tracking, the example finds that 80% of variation is explained by three factors: consistent wake time, completion of a morning braindump, and available morning resources. Those three then become the targets for improvement.

Application 2: Framework Selection

A useful framework explains much of the variation with few enough variables to use:

The expected-value model is more useful in this comparison despite lower theoretical accuracy. Four variables fit within working memory's seven-item limit, each can be manipulated, and together they retain the essential structure. A model that cannot be computed in the available capacity is harder to act on.

Application 3: Mental Model Evaluation

Ask whether the model reduces complexity while preserving predictive power. "Procrastination = activation energy exceeds available willpower" uses two variables, explains 70%+ of instances, and suggests interventions on either side of the comparison.

"Procrastination = psychological resistance" names the difficulty without specifying variables. It explains no particular outcome and gives no way to intervene.

This connects Occam's razor to compression: the simplest explanation that fits the data has identified the intrinsic structure that matters for prediction.

The Tracking Optimization Protocol

The goal is the smallest set of measurements that preserves the most explanatory power:

1. Initial phase (Days 1-30):
   - Track 10-15 variables (inputs + outputs)
   - Include everything potentially relevant

2. Analysis phase (Day 31):
   - Calculate correlations between all variables
   - Identify which inputs predict which outputs
   - Find clusters of correlated variables

3. Compression phase:
   - Keep 1 representative from each cluster
   - Drop uncorrelated variables (noise)
   - Result: 3-5 key variables capturing 80% variance

4. Validation phase (Days 32-60):
   - Track only compressed set
   - Verify predictive power maintained
   - Adjust if needed

5. Maintenance:
   - Continue with minimal set
   - Periodically check if intrinsic structure changed

The validation phase matters because the structure has to be discovered for this person and checked against later observations. The protocol does not assume that everyone has the same relevant variables.

Integration with Mechanistic Frameworks

Connection to Working Memory

The 4-7-item working-memory limit constrains how many dimensions can be processed at once. A problem with ten intrinsic dimensions exceeds a seven-item capacity.

There are three ways to handle the gap: externalize the work in a journal or on a whiteboard, find a smaller five-dimensional structure, or process subsets sequentially. Externalization lets the representation extend beyond biological working-memory capacity.

Connection to State Machines

State machines represent behavior as discrete states and transitions. Infinitely many next moments are possible, but roughly 5-10 default scripts organize actual behavior. The state machine is useful because those routines already supply a small structure to represent.

Connection to Expected Value

Motivation can appear to depend on mood, circumstances, personality, energy, time of day and recent events. The expected-value account identifies four variables that explain 90% of the variation: reward × probability / (effort × time_distance).

This formula describes the actual low-dimensional structure of the brain's motivation calculation. Its value comes from finding that structure, rather than merely being a convenient approximation.

Connection to Question Theory

Question Theory relates the cost of a search to its dimensionality. "What should I do?" leaves the search unbounded, described as O(∞). "What's next action on highest-priority task?" restricts it to one dimension, described as O(1).

A well-specified question narrows the search to the space where the relevant answer can be found.

Common Misunderstandings

Misunderstanding 1: "Simple = Simplified"

A simple model can be accurate because the phenomenon itself has few independent dimensions. When behavior has a three-variable structure, a three-variable model represents that structure without needing to be a dumbed-down account of something more complex.

Misunderstanding 2: "More Variables = More Accurate"

If five dimensions carry the relevant variation, recording 50 can make the model worse. The additional variables overflow working memory, introduce random correlations, obscure the pattern and reduce accuracy through overfitting.

Find the five that matter and examine them closely.

Misunderstanding 3: "Context Always Matters"

The manifold hypothesis implies that much of the apparent contextual complexity is decorrelated noise. If 90% of contextual factors have zero correlation with the outcome, they can be ignored for that decision. The relevant task is to distinguish signal from noise, rather than include every available detail.

Observable Patterns

Pattern 1: The 80/20 Distribution

A repeated observation is that 20% of variables explain 80% of the variation. The tracking examples include:

• Exercise and screen time explain 75%+ of sleep-quality variation.
• Wake-time consistency, a braindump and environment explain 80% of work-output variation.
• Sleep, exercise and social factors explain 70% of mood variation across 2-3 variables.

The distribution follows from lower-dimensional structure in real phenomena. It does not require a separate universal law.

Pattern 2: Compression Resistance Reveals Noise

Signal compresses when a few variables account for most of its variation. Noise requires many variables that each contribute little.

If reaching 90% accuracy requires 30 variables that each explain 3%, you are probably fitting noise instead of identifying the intrinsic structure.

Pattern 3: Framework Convergence

Independent frameworks arriving at similar small representations support the existence of a real underlying structure. Willpower has 3-5 main depletion sources. Activation energy has 2-3 main threshold factors. Expected value uses four variables, and state machines use 5-10 typical states.

The convergence reflects human behavior's roughly 3-10-dimensional intrinsic structure. Different useful frameworks discover it independently.

Anti-Patterns

Anti-Pattern 1: Premature Compression

Assuming "calories in vs calories out" before observing the system decides its dimensionality too early. A one-dimensional account might fit, or the actual structure might require three variables. Track the system first, then reduce the representation based on what you find.

Anti-Pattern 2: Forcing Arbitrary Dimensions

Choosing measurements solely because a theory recommends them can miss the relevant structure. Tracking macronutrients because "nutrition science says so" may not identify the variables that matter for this N=1 case. Begin broadly and let the observed correlations guide the selection.

Anti-Pattern 3: Ignoring Manifold Shifts

A three-variable model that worked for six months may stop working after a new job, a move or a relationship change. The relevant structure can change with the system. Periodically check that the model still fits current data.

Related Concepts

• Pedagogical Magnification matches analysis to intrinsic dimensionality.
• Working Memory limits how many dimensions can be handled at once.
• Discretization finds useful divisions in a process.
• State Machines represents behavior through discrete states.
• Tracking discovers which variables explain variation.
• Expected Value uses four variables to describe motivation.
• Question Theory bounds the space searched for an answer.
• Computation as Core Language develops the information-theory foundation.