unify docs

unify — Testing Strategy

Status: normative for the test suite Role: This document is the testing contract. It defines what "fully and correctly implemented" means as a set of machine-checkable conditions, and it exists because the previous suite proved that green checkmarks and a working product are independent variables unless something mechanical ties them together. Companion: docs/migration-plan.md (how we get from that suite to this one). The rule inventory lives at tests/conformance/rules.tsv; the gates are tests/conformance/check-traceability.mjs and tests/conformance/check-suite-hygiene.mjs — both runnable today.

1. Root-cause analysis: 93% coverage on a product that did not work

The suite this one replaced shipped with 240 files under tests/ (62 test files) and a reported 93%+ coverage while: unify init exited 1 and scaffolded nothing; automatic layout discovery did not exist; a page's content was silently deleted when the layout lacked the expected element while the build printed ✅ Build completed successfully!; <title> was replaced instead of merged; non-stylesheet <link> elements were stripped; --pretty-urls moved files without rewriting links; a missing <include> hung the build forever; a missing SSI include shipped the raw directive with exit 0.

All of these were re-verified while writing this document (2026-08-11, this machine). None of them is subtle. The suite could not see any of them, for five identifiable mechanisms. Every design decision in §2–§6 traces back to one of these.

M1 — The suite tested functions, not the product

Exactly one of 62 test files invoked the real CLI: tests/integration/fixtures-integration.test.js. Everything else imported internal classes and asserted on their return values. tests/unit/cli/commands/build-command.test.js opens with twelve assertions of the form expect(buildCommand.areaMatcher).toBeDefined() — constructor property checks. unify init, the first command every user runs, had zero test files; it could exit 1 forever without a single red mark. When the object under test is a function, a broken product with working functions is green.

M2 — The one real test had its assertion commented out

fixtures-integration.test.js did build fixtures through the CLI and did compare against expected output — and then, at line 374–375:

// For now, don't fail the test - just document the differences
// expect(comparison.valid).toBe(true);

The only end-to-end output comparison in the entire suite printed a warning and passed. The same file weakened its content expectations to match the broken implementation, with comments narrating the surrender: // Note: Third element may not be fully implemented yet, // Note: Full ID rewriting not yet implemented, // Note: Component scoping may not be fully implemented yet. When implementation and test disagreed, the test was edited. A fixture tree with expected outputs existed (tests/fixtures/full-site/expected/) that no test read at all; it was deleted with the legacy suite.

M3 — The suite ratified bugs as contracts

tests/unit/core/cascade/head-merger-fixes.test.js asserts expect(merged.title).toBe('Page Title Wins') — the title-replacement behavior that loses the site name on every page was written down as the expected behavior, named approvingly in the test data, and locked in. tests/integration/build-success-validation.test.js:116–120 asserts that a build with a broken image reference succeeds (expect(result.success).toBe(true) alongside expect(result.warnings[0]).toContain('missing-image.jpg')). Silent failure wasn't missed by the suite; it was specified by the suite. There was no external authority (a spec with rule identity) that a test could be checked against, so "what the code does" became the oracle.

M4 — Coverage measured execution, not verification

Five test files carry "coverage" in their names (html-processor-coverage.test.js, html-processor-focused-coverage.test.js, security-scanner-branch-coverage.test.js, short-name-resolver-coverage.test.js, markdown-processor-yaml-coverage.test.js) — tests written to make a number go up. The unit suites contain 322 assertions of the form toBeDefined() / toBeTruthy() / not.toThrow(): assertions that execute code and verify nothing about its output. Coverage counts lines entered; a test that calls processLayout() and asserts the result is an object covers every line of a wrong answer. And the inverse hole: src/core/layout-resolver.js — automatic layout discovery, the product's marquee convention — is 464 lines, 412 of them commented out, with zero importers. A file nothing imports never enters the coverage denominator, so deleting a feature's wiring raised the metric. Coverage can neither see wrong output nor missing features. It measured the suite's enthusiasm, not the product's correctness.

M5 — Comparisons were normalized until they couldn't fail

The one output comparison that existed ran both sides through normalizeHtml() — replace(/\s+/g, ' '), collapse everything — before comparing. Collapsing all whitespace erases differences inside text content, attribute values, and <pre> blocks — real bug classes for a composition engine — and it was applied ad hoc, per test, wherever an assertion was inconvenient. Weak comparators are how "close enough" ships. The countermeasure is not "never normalize" — it is one narrow, stated comparator (§2), implemented once in the harness: normalization is confined to the single difference class the spec itself declares non-normative (whitespace between block-level elements, conformance spec §3), and every other normalization, anywhere, fails hygiene rule H5.

Summary of the disease: no single observable-product oracle, no external rule authority, a metric that rewards execution over verification, and a cultural pattern of weakening tests to match code. Every one of these has a mechanical countermeasure below; none of the countermeasures is a policy or a promise.

2. The tier model

Observable build output is the primary object of testing. Tiers are numbered by authority: when two tiers disagree, the lower number wins, and the higher tier is what gets fixed.

Tier 0 — Golden path E2E (the product works)

Drives the installed entrypoint (bun src/cli.js, and the compiled binary in the release job) exactly as a user would, with subprocess spawns, real temp directories, a hard timeout on every invocation (a hang is a failure — M-item: the earlier codebase's missing-include hang), and no imports from src/**:

Proves: the five-minute site is real; the commands users type work end to end. Cannot prove: rule-by-rule correctness — a golden path can be green while an edge rule is wrong. That is Tier 1's job.

Tier 1 — Conformance fixtures (the spec is implemented)

The heart of the suite. A generic harness (one file, ~200 lines) iterates fixture manifests; a fixture case is: a source tree, flags, and a declared outcome — expected output tree (tree-exact per the comparator below, bidirectional), expected diagnostics (exhaustive), expected exit code, expected publish state. The harness spawns the CLI; there is no per-case test code to weaken. Three sets ship today:

Proves: each normative rule, including exact output and exact diagnostics, against the real CLI. Cannot prove: time-dependent behavior (watch), long-running processes (dev), or that the rules compose over arbitrary sites — Tiers 0 and 2 and the property harness cover those.

Tier 2 — Engine-contract tests (the operational promises)

Targeted behavior tests, still through the CLI, still mock-free, for the contracts a static fixture can't express: transactional publish sync (unchanged files keep inodes/mtimes, stale outputs deleted, temp-then-rename — PUB-02), --dry-run writes nothing and reports each page's inputs (PUB-04, DRY-01..03), watch coalescing / watch-output ≡ fresh-build equivalence over a scripted edit sequence (WCH-01..04), dev server injection scoping (WCH-05..06), unify.yaml precedence (CFG-01..03), exit-code taxonomy including exit 2 cases (DIA-04), determinism (two runs → identical bytes on stdout, stderr, and tree — DIA-05), DEBUG=1 (DIA-09).

Tier 3 — Unit tests (developer scaffolding, zero authority)

Unit tests on pure internals (the slugger, the glob matcher, splice-span arithmetic) are welcome for development speed and may use whatever test doubles they like — but they carry no conformance authority: they cannot declare rule coverage (covers() is rejected outside tests/conformance + tests/e2e by the hygiene gate), they don't gate release, and when a unit test disagrees with a fixture, the unit test is wrong by definition. This inverts the previous suite, where unit tests were the majority authority (M1) and ratified bugs (M3).

Where a new rule's coverage should land first (measured, 2026-08). The first mutation sweep of the newer discovery-and-evaluation modules found three genuine gaps, and their distribution was the lesson: all three sat in feed.js, the one of those modules with no unit test file, while report.js — which has one — had no real gap at all (its existing unit test kills the fingerprint mutation in 0.4 ms). The same defect costs ~3 s to detect through a CLI spawn and ~1 ms through the module's exports; the unit twins of all three gaps (tests/unit/core/feed.test.js) kill their mutations in ~160 ms of whole-file time. So the division of labour is: one conformance test per rule proves the wiring — the flag reaches the module, the module's output reaches the tree — and the edge-case matrix belongs here at Tier 3, where the next input costs a millisecond instead of a spawn. Authority is unchanged: when the tiers disagree, Tier 3 is wrong by definition — the tier that finds a bug first is simply allowed to be the cheap one. And the wiring test is not optional, for a reason the same session supplied: --generate shipped parsed but never mapped through cli.js — a complete no-op — and every unit test of the generator function would have stayed green while the feature did not exist.

Comparator discipline (all tiers): tree comparison is bidirectional (an extra emitted file fails, a missing file fails) and goes through exactly one comparator, implemented once in the harness (tests/conformance/compare.mjs). Its contract, in full: non-HTML files are compared byte-for-byte. HTML files are parsed and compared exactly on the file set, the doctype, element structure, tag names, attributes (names, values, and order), comments, and text content — with precisely one normalization: a text node consisting entirely of whitespace, whose parent is not <pre>, <textarea>, <script>, or <style>, is dropped from both sides before comparison. That is the conformance spec §3 waiver ("whitespace between block-level elements is not normative") and nothing more. A text node containing any non-whitespace is compared byte-for-byte, including its internal and surrounding whitespace — whitespace inside text-bearing content is significant. One blind spot, named rather than hidden: the comparator cannot tell a block gap from an inline gap without a tag taxonomy, so a whitespace-only text node between inline siblings (<a>x</a> <a>y</a>) is also normalized; where an inline gap is itself the thing under test, the fixture must make it text content (give the siblings a text-bearing parent with surrounding text). Trimming, entity folding, attribute reordering, tag-case folding, and any other normalization are forbidden; ad-hoc normalization anywhere else in a behavior test fails hygiene rule H5 (the M5 countermeasure, narrowed and stated rather than absolute). Diagnostics are parsed on the stable FILE:LINE: SEVERITY: prefix; message prose beyond declared substrings is not asserted (the spec says prose is not contract — tests must not fossilize it). stdout/stderr comparisons (DIA-05, G6) remain byte-exact: determinism of one implementation is a byte-level claim even though cross-checking output trees against fixtures is not.

3. Spec-rule traceability, mechanically enforced

This is the mechanism that makes "fully implemented" a measurable claim.

3.1 The rule inventory

tests/conformance/rules.tsv — one row per normative claim in docs/conformance-spec.md, extracted section by section. It held 202 rows (199 gated + 3 structural) at the last count written down here, and grows with the spec; the checker prints the current count, which is the number to trust. IDs are stable, never reused, and namespaced by area, reusing the spec's own labels wherever the spec numbers things. Retired IDs stay retired: LAY-06–LAY-08, HED-08, P06, and FIX-08 died with layout chaining; A05–A07 were merged into A13 (the duplicated-construct advisory); A03, A04, and A15 were retired by the ratification rounds (A04 became problem P20); no test may reference them.

| Prefix | Source | |---|---| | PIP-* | §2 pipeline order & best-effort | | S01–S12, SHL-01 | §3 splice rules + shell rule | | EXC-* | §4 classification/exclusion/never-shipped/copy | | INC-* | §5 includes | | LAY-* | §6 layout resolution (incl. the no-chaining problem) | | MRG-* | §7 composition (incl. the no-nested-slots problem) | | HED-01–07 | §8 head-merge table rows | | ATT-* | §9 root attributes | | MD-* | §10 Markdown | | URL-* | §11 URL phases | | REF-* | §12 reference check | | COL-* | §13 collisions | | DIA-* | §14 diagnostics contract | | P01–P21 (P06 retired) | §14.2 closed problem list | | A01–A14 (A05–A07 merged into A13; A03, A04, A15 retired) | §14.3 closed advisory catalogue | | PUB-* | §15 transactional publish | | WCH-* | §16 watch/dev | | DRY-* | §17 dry-run report | | CFG-* | §18 unify.yaml | | SCF-* | §19 scaffold contract | | FIX-01–14 (FIX-08 retired) | the spec's worked examples | | MAN-* | §20 the final-output page manifest | | SIT-* | §21 sitemap generation |

Counts are deliberately absent: this table drifted twice in three commits while the checker's own printed totals stayed correct by construction. check-traceability.mjs prints the current inventory size, the covered count, and the gap list on every run — that output is the number to quote, and it cannot go stale.

A row's testkind is fixture, targeted, e2e, or structural. The three structural rows (MRG-18 the content-loss law, COL-04 no-last-write-wins, WCH-07 the closed dev scope) are invariants asserted by the shape of the whole suite (exhaustive diagnostics + full expected trees + closed CLI test) rather than by one test; they are exempt from the per-rule gate and documented as such in the TSV.

3.2 How tests declare coverage

Two declaration channels, both machine-read:

  1. Fixture manifests (manifest.json per set): each case carries "rules": ["S03", "MRG-09", …]. Because the harness iterates the manifest, a manifest row is a test — declaring without running is impossible by construction.
  2. covers() in targeted/E2E tests: behavior tests call covers("WCH-02", "PUB-02") (a helper exported by the harness) inside the test body. The call does two things: it records the declaration for the static check, and at runtime it appends {rule, test, status} to the conformance ledger (.conformance-ledger.jsonl) when the surrounding test passes.

3.3 The gap check

tests/conformance/check-traceability.mjs, two modes, both exit 1 on failure:

Status (measured 2026-08-13, --static): 202 rules (199 gated, 3 structural); every gated rule covered — baseline.txt empty, so the phase gate and the release semantics were the same check. All thirteen FIX rows realized, the §7 spec-bug set (B1–B7) and the four pinned readings closed.

baseline.txt was non-empty again for a stretch, and is now empty: unify audit --format json (§31.1) publishes pages as the whole record, so the last seven rows — MAN-02/03/04/07/08/09/10 — became observable through the real CLI at once and are covered by tests/conformance/manifest-observable.test.js. What follows is the reasoning that put them there, kept because the pattern recurs. Conformance-spec §20 (the final-output page manifest) is an implementation boundary: product-spec §6.2 forbids exposing it as an author-facing format, so a rule about it can be specified before any CLI surface exists for a behavior test to observe. Those rows are baselined rather than claimed, and each is closed by the consumer that makes its field observable — the sitemap closed MAN-01/05/06 in the commit that landed it. Baselining is the honest record of "specified, not yet observable"; claiming coverage from a Tier-3 unit test would not be, since §3.2's declaration channels deliberately read only tests/conformance and tests/e2e.

The release semantics are unchanged and still blocking: release.yml runs --runtime with no baseline at tag time, so a release cannot ship until the file is empty — which it now is, so the phase gate and the release semantics are once more the same check. The checker's output is the authoritative gap list at any moment (see docs/cicd-workflows.md for which job means what).

3.4 Why declaration ≠ vacuous claiming

A declared rule could still be weakly tested (the ID is on a case that only brushes the rule). Three structural mitigations: fixture cases assert full expected trees and exhaustive diagnostics, so a fixture cannot assert "nothing" — there is no weak form of tree equality under the §2 comparator; the manifests carry ruleNotes naming which branch of a rule each case pins, and branch gaps found in review become new landmine cases (cheap: a directory and a manifest entry); and the two-sided landmine convention — every behavior has a firing case and an adjacent builds-clean case (include-depth-10/-11, collision-pretty-landing/-noflag, working-format/-strict) — kills tests that pass by matching everything.

4. The metric replacement

The release metric is the conformance ledger, not coverage: every gated rule in rules.tsv recorded green, by tests that ran, against the real CLI, in this run — plus the tree-exactness and exhaustive-diagnostic discipline that makes those recordings meaningful. This number has the property coverage lacked: it cannot be raised by executing more lines, deleting wiring (M4), or weakening assertions (M2); it can only be raised by making a spec rule demonstrably true.

What replaces the dead-code blindness: a reachability check — every file under src/ must be reachable in the import graph from src/cli.js, or the build fails (bun build --target=bun src/cli.js + tree walk; release gate G8). layout-resolver.js — 464 lines, 412 commented, zero importers, invisible to coverage — becomes structurally impossible to ship.

What coverage is still for: a diagnostic, reported on every CI run, never a gate, never quoted in the README. Two legitimate uses: (a) inside a new core module during development, un-hit branches are cheap hints of missing landmines — the response is a new fixture, not a new unit test; (b) a sudden drop on a PR is a code-review signal that new code has no behavioral consequence the suite can see. Both uses feed Tier 1; neither blocks or green-lights anything. No threshold number exists in this strategy on purpose: any threshold reinstates the incentive that produced html-processor-coverage.test.js.

5. Anti-regression rules for the suite itself

The defect is rarely in the function; it is in what the function was handed. Six rounds of independent review on the newer discovery-and-evaluation modules produced the same shape every time, and it is worth stating before it has to be rediscovered. isSelfCanonical was correct and its caller's boolean fold was not. validateAnchors was correct and the row set its caller passed was not. stripBaseUrl's comparison was correct and the normal form its result shared with parseBaseUrl was not. In each case the first repair added a test for the callee, the suite went green, and the defect stayed.

The fixes that held all did the same thing: they deleted the parameter rather than testing around it. anchorProblems takes no row set, because a caller that can narrow one is a caller a test of the callee cannot pin. The mutation sweep has no environment variable, because an ambient switch is a parameter every shell can set. parseBaseUrl and stripBaseUrl share one normal form, because two spellings of one prefix is a parameter with no owner. Where a wrong value cannot be supplied, no test is needed to catch it being supplied.

The corollary for this section: a guard is pinned at the level its defects live, not at the level it was convenient to extract. When adding one, revert each defect it was written for and confirm the suite goes red — if it stays green, the guard is one layer too shallow.

Enforced by tests/conformance/check-suite-hygiene.mjs (in CI as gate G9), which greps behavior-test sources (tests/conformance/**, tests/e2e/**) for H1–H5 and shipped source (src/**) for H6; H1–H5 are each the countermeasure to one of the mechanisms from §1, H6 to an incident of this repository's own review protocol:

The suite refuses to start on a tree it cannot parse, and bounds its own runtime. Both are bunfig.toml [test] preload entries, because they must run before the runner loads a test file — no test can ask either question from inside the run, which is the §8 hang stated as a design constraint. tests/preflight.mjs hands every JavaScript file the suite can load to Bun.Transpiler.scan() — parse, never import: importing to find out would run module side effects on a tree already known to be untrustworthy — and exits 2 with a located message about 40 ms in. The set it checks is a sweep of src/** and tests/** plus that sweep's import closure: a sweep cannot miss a file the way a graph walk can, and the closure only ever adds, so a specifier form the walker does not recognise can never remove a file from the check. fixtures/ is swept out and re-entered only through a real import, so tests/fixtures/landmines/runtime-cases.mjs is covered while a deliberately malformed fixture asset is not — unify ships broken scripts byte-for-byte, and a guard that refuses to start on a test case is a guard that gets switched off. tests/watchdog.mjs is loaded first and has no imports at all, so it still bounds the run when the parse gate or its walker is itself the unparseable file; it arms one unref'd timer for UNIFY_TEST_BUDGET_MS (default 600 000 ms, ~7× the full suite; a malformed or empty value falls back to the default rather than firing at once) and exits 3. Raise the budget for a genuinely slower machine, never to silence it — it is the one ambient switch this suite allows, and only because no value of it means "never". Neither guard touches bun src/cli.js: preload under [test] applies to the test runner only. Both are also the only files outside src/** that mutations.tsv may name — they are code the suite has to prove it notices, whereas mutating an ordinary test would score the sweep against its own assertions. And both publish what they did on globalThis (the budget they armed; the file list they parsed), because every other test of them runs in a scratch project carrying copies: without that marker a deleted preload line leaves the guards demonstrably working and the real suite unguarded.

tests/module-graph.mjs is the single owner of which files this tree's JavaScript consists of and what each one pulls in — check-module-graph.mjs (G8) and the preflight both ask it. They had a copy of the walk each, which is §5's own shape one level up: a specifier form one copy missed was invisible in the other.

A duplicated document needs a gate, exactly as duplicated code does. README.md embeds docs/authoring-rules.md verbatim between two marker comments, because product-spec §6.7 requires one rule set rather than tool-specific variants. Nothing checked the copies, and they drifted on the first edit that mattered: §26 gave schema: a meaning, the rules file gained it, and the README went on telling readers that JSON-LD has no frontmatter key. A marker pair is a promise that something copies one into the other, and until tests/unit/docs-sync.test.js existed that something was whoever remembered. The comparison is byte-exact on purpose — "nearly the same" is the state it exists to catch, since two sentences differing by one clause is precisely how a reader ends up with the wrong rule and no way to tell which copy is current. The same file pins the 60-line budget the document's own name claims, so a feature that needs a paragraph has to argue for it rather than append one.

A reviewer reports which tests a mutation KILLED, not that a mutation was run. Four rounds of independent review on the page-manifest work found six defects, and every one had the same shape: a rule written correctly in the conformance spec, implemented differently, and a suite that stayed green either way. Coverage cannot see that class — the line executed, it just did the wrong thing and nothing asserted otherwise. tests/conformance/mutations.tsv names one anchor→replacement pair per rule with the rule it defends, and bun tests/conformance/run-mutations.mjs applies each in a throwaway copy of the tree, runs the suite, and fails on any mutation the suite does not notice. A survivor is not a bug in the code — it is a rule the tests cannot distinguish from its opposite, which is the state that let all six through.

A green sweep is not a statement about the inventory. The file defends the rules its rows name — a fraction of the inventory — and nothing else. "Every mutation killed" means those anchors are pinned; reading it as "the spec is pinned" would be a new way to reach the same false confidence this check exists to remove.

Flakiness is measured, not listed. The baseline runs twice and the two failure sets are compared: a test that differs between two runs of identical source is flaky by observation; one failing in both is genuinely red and aborts the sweep. An earlier version matched test names against a regex, which excluded every test in watch-dev.test.js — including the deterministic filesystem assertions — so a dev-server mutation was credited to a Tier-3 unit test while the authoritative behaviour test that also caught it was filtered out. That inverts §2's authority order, and the only available "fix" for the next such row would have been to weaken the regex further. Measuring also means one environmental flake no longer discards a fifteen-minute sweep, which is the habit that trains people to re-run a check until it passes.

The runner refuses to start unless the suite is green unmutated, because its first version did not: it treated any non-zero exit as a kill, so on a red tree every mutation reported KILLED and the sweep exited 0 — the answer inverting exactly when it mattered most. A reviewer found that within a day of the file landing, and the decision logic is now exported and unit-tested against that exact scenario. A kill must name a test that was passing before; a non-zero exit with no failing test is CRASHED, so a malformed replacement cannot pass as evidence; and a hang is TIMEOUT, never a kill.

It is deliberately not a release gate, and — recorded here because this sentence used to promise the opposite — it does not graduate to one now that the phase baseline is empty. The full-inventory sweep that milestone imagined was run once, and its economics settled the question: all 152 pre-existing rows re-passed, at a cost of hours, while every finding came from the handful of rows written against code that was days old. The value concentrates entirely in fresh and just-repaired modules, so the standing policy is: sweep the prefixes you touched, at review time (bun tests/conformance/run-mutations.mjs feed-), never the whole inventory. The runner targets each row at the test files the coverage ledger maps to its rules and escalates any survivor to the full suite, so a targeted module sweep is minutes and a wrong or stale ledger costs time, never correctness. When a sweep does find a survivor, land the killing test at Tier 3 where the rule is logic and at the conformance tier only where it is wiring (§2). The sweep's output belongs in the review report. It never touches the working tree, for a reason this repository has already paid for twice: an in-place mutation was swept into a commit by an unrelated git add -A, and a second in-place run silently reverted a real fix when its restore step did not execute after a timeout.

Never mutate anything you need back. Three separate incidents in one review cycle had one root cause — a restore step that could be skipped:

  1. A reviewer's mutation was live in the working tree when an unrelated git add -A ran, and shipped to main with a real check deleted. The suite passed: nothing executed that branch.
  2. An in-place mutation loop timed out mid-iteration, so its restore never ran, silently reverting a fix made minutes earlier.
  3. A reviewer's extracted baseline copy was overwritten with a newer revision's file. Ten minutes of before/after comparisons were invalid, and it was caught only because one result was impossible — the "before" commit exhibiting the "after" behavior.

The durable fix is structural, not procedural: a comparison must not depend on restoring anything. run-mutations.mjs embodies it for the working tree — it writes only inside a throwaway copy, so a crash mid-run can damage nothing. The same discipline extends to every artifact a comparison rests on:

There is a fourth incident with the same root cause one level down, and it is worth stating because the fix people reach for first does not work. The throwaway copy is ~1 GB, and the sweep deletes it in a process.on("exit") handler — which covers every ending the program controls and none of the endings that actually happen: SIGKILL, an OOM kill, a timeout that escalates, a container restart. Five orphaned copies (~5 GB) accumulated on a fixed disk allowance and slowed a live sweep to a crawl, and the symptom looked like a slow machine rather than a full disk until somebody ran df. A cleanup that only runs on the paths that were already fine is not a cleanup. The one that works runs at startup, on the evidence the previous failure left behind: each run drops its pid in its own work directory, and a new run removes any sibling whose pid the operating system says is gone. Liveness is asked of the OS (process.kill(pid, 0)) rather than inferred from an mtime, so a genuinely long sweep keeps its directory for as many hours as it needs — a promise an age threshold cannot make.

Process rules that cannot be fully mechanized, stated as review law with their partial mechanizations:

6. Release gates

The release ships when a clean CI run on the release commit satisfies all of the following. Each is a command with an exit code; no gate is a judgment call:

| # | Gate | Check | |---|---|---| | G1 | Suite green | bun test exits 0 (includes Tiers 0–3; every behavior test under its hard timeout, and the whole run under the §5 watchdog budget) | | G2 | Traceability | check-traceability.mjs --runtime .conformance-ledger.jsonl exits 0: every gated rule recorded by passing tests; zero unknown IDs; spec↔inventory sync clean | | G3 | Kitchen sink | all four kitchen-sink profiles tree-exact (§2 comparator) / publish-state-exact | | G4 | Landmines | every declared diagnostic fires with declared severity/location; zero undeclared diagnostics suite-wide; both publish-block sentinels byte-untouched | | G5 | Golden path | all five init templates: scaffold → build --dry-run --strict exit 0 → build → reference-clean output; dev smoke passes | | G6 | Determinism | two consecutive builds of kitchen-sink: identical output trees, identical stdout bytes, identical stderr bytes | | G7 | Watch equivalence | scripted edit sequence under watch yields a tree byte-identical to a fresh build; a no-op save rewrites nothing (mtime check) | | G8 | No dead modules | check-module-graph.mjs exits 0: every src/** file reachable from src/cli.js in the static import graph | | G9 | Suite hygiene | check-suite-hygiene.mjs exits 0 | | G10 | Docs lockstep | docs/authoring-rules.md ≤ 60 lines and linked from the README, whose own summaries of it stay consistent (product-spec §7, item 5) | | G11 | Binary parity | the compiled Linux x86_64 binary passes the Tier-0 golden path; Linux ARM64 and macOS x86_64/ARM64 release binaries build successfully (parity runs where runners exist) | | G12 | Runtime parity | node src/cli.js passes the Tier-0 golden path for all five templates and emits a scaffold, a dist/, a stdout and a stderr byte-identical to the bun src/cli.js run; needs a node >= package.json's engines.node on PATH, and fails rather than skips without one |

Coverage is reported alongside the gates for the record. It gates nothing.

7. Spec bugs and pinned readings found while building the fixtures

Writing exact expectations is the strongest spec review that exists. Filed here rather than papered over; each needed a spec amendment (with its inventory row updated in the same commit, which the sync check enforces). All are now closed — the entries below record each ruling and the fixture that pins it:

8. What this strategy does not cover, said plainly

Performance (no perf gates until real sites are slow — a product non-goal), fuzzing (a future property harness — "random tree in, law holds" — would strengthen MRG-18 beyond fixtures; not required), Windows/macOS binary behavioral parity beyond the smoke run (G11 runs the full golden path on Linux only until CI runners exist for the rest), and browser-preview parity for the deferred polyfill (its "build and polyfill must agree" check belongs to the polyfill's own milestone).

The one known hole, closed 2026-08-19: bun test used to hang instead of failing when a module the suite loads cannot be parsed. Tier 0 says a hang is a failure, and this was the one place it was not. The cause is not in any unify code: on bun 1.3.11, when two or more test files fail to load because a module they import cannot be parsed, the runner prints "Unhandled error between tests" and then spins at 100% CPU with no child processes and no exit — measured past 900 s, against ~85 s for the healthy suite. One such file exits 1 in under a second; two never exit. The spin happens between test files, so no per-test timeout is in reach, and the runner prints nothing for a passing file, so the stall does not even look like one. A single syntax error in src/core/urls.js clears that threshold on its own, because much of tests/unit/** imports it transitively; so does one in tests/conformance/support.mjs. The reproduction recorded here previously blamed tests/conformance/watch-dev.test.js and its un-timed fetch calls. That was wrong, and re-measurement is what showed it: with a broken urls.js that file fails 5/5 and exits in 19 s, tests/conformance as a whole finishes in 61 s, and during the hang there are no child processes at all — a diagnosis nobody had re-run, which is its own lesson about a hole recorded once and left. The closure is tests/preflight.mjs and tests/watchdog.mjs, described in §5; both broken trees above now exit 2 in under a second. What remains open is bun's own bug: if a later version stops spinning, the preflight becomes belt-and-braces rather than load-bearing — it still turns a suite-wide load failure into one located line — and this paragraph should be re-measured at the next bun bump.

It also does not test whether the documentation is any good. Every tier here checks that the engine implements the spec; none of them checks that a person handed docs/authoring-rules.md can build a site from it. That is a different experiment with a different failure mode — a rule can be correctly implemented, correctly specified, and still worded so that nobody follows it — and it has its own procedure in docs/ratification-protocol.md: agents author from the rules alone in isolation, and each failure is triaged into documentation, specification, implementation, or outlier. Two spec defects in this document's own catalogue were found that way, both because several independent samples made the identical "mistake" and the spec turned out to be the thing that was wrong.