Lockfile
What It Is
A lockfile is the versioned bundle that pins what "good" means in a domain: the skill, the golden set, the verifier, and the rejection ledger, shipped as one forkable unit. The name is borrowed from software packaging on purpose. A manifest declares intent loosely — "some recent version of these libraries" — and produces a different build on every machine; the lockfile pins exact versions so the build reproduces anywhere, byte for byte. A skill file alone is a manifest: intent, loosely stated, reinterpreted on every run. The four-piece bundle is the lockfile: it pins exactly what "good" means at this moment, reproducibly, so quality ships as checkable invariants instead of vibes.
The division of labor with its sibling article: taste compilation is the compiler — the loop by which felt verdicts climb through language into machinery. The lockfile is the artifact that loop emits. The reusable unit of value in a generation system is not the prompt, not the model, and not any output artifact; every one of those is either rented or depreciating. It is this bundle — the only object in the pipeline that is simultaneously owned, versioned, and executable by whatever model shows up next year.
The Four Pieces
| Piece | What it is | What it does | Failure if missing |
|---|---|---|---|
| Skill | The procedure and sequencing logic | Tells the agent how to work | No process — every run improvises |
| Golden set | Curated exemplars that define quality ostensively | Shows what "good" is, since taste can't be fully specified in words | Instructions without taste — the wheelwright's son |
| Verifier | A mechanical gate; quality as checkable invariants | Makes agents checkable instead of merely careful | Quality regresses to vibes |
| Rejection ledger | What failed, on what axis, kept verbatim | Preserves the negative space; the intensity of the original rejection is itself signal | Every failure gets re-discovered |
The rule that follows: never ship or store a skill alone. A skill without its golden set is instructions without taste. Without its verifier, quality is unenforceable. Without its ledger, the system has amnesia about its own failures. Version the four together so the definition of "good" evolves as one coherent object.
The ledger deserves special defense against tidiness: keep rejections verbatim. "This felt cheap and lazy" carries information that a sanitized "insufficient polish" destroys — the heat of the original reaction is part of the data. The ledger is an accrual substrate in miniature, and the whole bundle is memory as the system: the skill is a temporary execution; the pinned taste is the persistent state that actually constitutes the pipeline.
Why Distilled Rules Fail
The default instinct for institutional memory is distillation: boil the lessons down to clean rules, file the rules, discard the mess. Will's verdict on that instinct, after watching a craft corpus fail to transfer through its own rulebook:
"The distilled rules are TRASH. Never give distilled rules — we need ANNOTATED EXAMPLES. ... Never give pure rules. Always give examples — and counterexamples. Explain the examples."
The mechanism of the failure: a rule is a compression whose decompression key is the taste that wrote it. To its author, "no unjustified runs" is a handle attached to a dozen remembered instances; to everyone else — a new hire, an agent, you in six months — it is an underdetermined string that admits every mistake it was written to prevent. The examples are the payload. The rule without its annotated instances transfers the words and strands the judgment.
There is a second, subtler corruption: letting the generator author its own exemplars. Three rounds of a failed register document produced the law: AI-written examples don't teach the register at all, and the version that finally held quoted the target corpus verbatim and derived its observable properties from what was quoted.
The law is the corpus. Golden examples must be actual members of the target distribution, not a model's impression of that distribution — an AI-authored exemplar imports the exact register the gate exists to keep out, then certifies it as the standard. Point at the real thing; derive properties from what you pointed at; never author the target.
The Example Is the Specification
If examples are the payload, they must be treated with a specifier's care, because whatever structure the exemplars actually exhibit is the standard the system will reproduce. Ten golden examples with a specific shared property specify that property; vague exemplars specify vagueness. The compression of the whole methodology:
"Examples transfer taste, constraints mechanize truth, and selection pressure is what keeps the dataset valuable."
That sentence assigns each piece of the bundle its channel. The golden set carries what cannot be articulated — taste defined ostensively, by exhibits, which is the loophole the wheelwright parable leaves open (see taste compilation for the parable; this bundle is what his shop never had). The verifier carries what can be made mechanical — invariants, gates, truth conditions. And the ledger's counterexamples carry discrimination: showing what bad looks like, annotated with why, teaches the boundary in a way no list of virtues can. A golden set without counterexamples defines a target with no edges.
Start From the Source, Verbatim
A corollary about how the golden set is used, not just built. When extending complex work, do not ask a generator to re-derive from a high-level description — start from the existing source verbatim and iterate by editing:
"For complex outputs with lots of hidden scaffolding, the right pattern is start from the existing source code verbatim, then iterate by editing. Don't ask it to rederive the whole thing from scratch — too much supporting structure lives implicitly in the code."
The codified lesson, after a re-derivation destroyed a week of tuning: "re-deriving tuned constants isn't caution, it's quality destruction." A converged artifact carries invisible load-bearing structure — constants, orderings, phrasings that encode dozens of resolved verdicts. Regeneration discards all of it and re-rolls the dice. The same physics from the constructive side:
"A good starting point is a really, really underrated thing — to iterate you need something plausible that will get you there. Starting from scratch, you have to go back all the way to Noah's Ark."
A pile of negative directions is not a starting point; one positive example is. This is what the golden set is for operationally — not a museum of past quality but the seed every new run forks from.
Constraint + Driver
A lockfile that pins only constraints produces work that is correct and dead. The bundle must carry both halves of a two-part engine:
"The juice isn't JUST in the DSL — the DSL is CONSTRAINING, but the PROMPT is what drives it to explore and imagine."
The constraint surface (grammar, verifier, components) defines what cannot go wrong; the generative driver (the prompt, the ambition setting) is where creative temperature enters. Neither alone produces quality: constraints without a driver converge on the safest legal output, and a driver without constraints produces vivid garbage. The storage consequence: store (prompt, context, output) triples, not artifacts alone. An output without the driver that produced it is a result you cannot rerun — you pinned the answer and lost the question.
Versioning and Forkability
Pinning makes the bundle reproducible; versioning makes it evolvable — every promote to the golden set and every ledger entry is a commit to the definition of "good," and the definition evolves as one coherent object instead of drifting piecewise.
And it is forkable, which is the under-appreciated property. Taste transfer between people has historically been apprenticeship — years of co-located pointing, the only channel available for judgment that cannot be specified. A versioned bundle changes the transfer economics: the grammar and verifier move losslessly, and only the golden set needs re-convergence against the recipient's own judgment. Someone else takes the bundle, re-selects the exemplars against their own eye, and inherits the rest for free. The apprenticeship compresses to the part that was actually personal.
The Readiness Meter
One instrument is still missing from most bundles: a readiness meter. "Am I ready to fan out?" is a measurement, not a vibe — silently run ten candidates, score them against the gates and the golden set, and report P(good | ask). The canonical day-1 failure — "they all kind of look like shit" — was exactly this number sitting at roughly 0/10, unmeasured. Fanning out at P ≈ 0 produces ten variants of garbage; the meter tells you whether the pinned language holds before you spend the batch. A lockfile with a readiness meter is self-certifying: it can report whether it is sufficient for the domain before you bet a run on it.
Integration with the Mechanistic Framework
Connection to Taste Compilation
The compiler that emits this artifact. That article owns the process — verdicts climbing through language into machinery, write-back on every selection event; this one owns the output format. Process and artifact, one mechanism apart.
Connection to Selection over Design
Architect mode's actual deliverable. The architect phase ends not when the system works but when the bundle exists; garden mode is then a read-write loop against it — every promote updates the golden set, every prune updates the ledger.
Connection to Accrual Substrate and Memory Is the Substrate
The ledger and golden set are accrual substrates: every verdict visibly deposits, nothing is re-derived. And the bundle is the substrate-as-system claim made concrete — the skill is a temporary execution; the pinned, versioned taste is the persistent state that actually constitutes the pipeline.
Connection to Ratchet
The lockfile is the judgment lock in the lock taxonomy: the structure that holds compiled verdicts so the next run cannot regress below them. Version by version, it is the pawl on the quality wheel.
Connection to Pedagogical Magnification
Annotated examples and explained counterexamples are magnification applied to taste transfer: showing the work at the resolution where the judgment actually lives, instead of shipping the compressed conclusion.
See Also
- Taste Compilation — the compiler; this is what it emits
- Selection over Design — architect-then-garden, with the bundle as the hand-off object
- Accrual Substrate — the ledger as a substrate where every verdict deposits
- Memory Is the Substrate — the pinned bundle as the system's persistent state
- Ratchet — the general retention law; the lockfile is its judgment instance
- Compounding Artifacts — what survives model churn, and why this bundle is the compounding one
- Pedagogical Magnification — why annotated examples teach and distilled rules don't
- Structure over Request — the verifier as quality enforced by topology
- Error Signal — the loop whose closed errors the lockfile pins; golden set, verifier, and rejection ledger as cached signals
Core Principle: The reusable unit of value in a generation system is the versioned four-piece bundle — skill, golden set, verifier, rejection ledger — pinned as one unit the way a lockfile pins dependencies, so "good" reproduces as checkable invariants instead of vibes. Never ship or store a skill alone. Distilled rules are trash as institutional memory: the example is the specification, counterexamples teach discrimination, and the law is the corpus — never AI-authored exemplars. Store constraint and driver together as (prompt, context, output) triples; start from the source verbatim instead of re-deriving; keep the ledger's heat. Then the bundle is reproducible, forkable, and worth more than anything it produces.
A skill without its golden set is instructions without taste; a rule without its example is a compression nobody can decompress. Pin all four, and "good" survives you, your model, and your mood.