unify docs

Integrating compiled components: the compile-to-asset pattern

Role: The recipe for using a component framework — Svelte here, but the shape is the same for anything with a compiler — on a unify site, without adopting a framework for the site. unify needs to know nothing about the framework, and that is the design: you compile the component to an ordinary JavaScript file before unify build, and unify ships it like any other asset. Every literal in this document is tested; the worked example is examples/forge-svelte. (That example pins Svelte 5 and uses the recipe's own mount() call; the three-file shape is identical.)

The contract, in four lines

The Svelte recipe

The component — say components/FeeCalculator.svelte at the project root, maintained by whoever writes your Svelte — needs three small files and two commands. Everything below runs from the project root, the directory holding site/, scripts/, components/ and package.json. Once:

npm install svelte esbuild esbuild-svelte

scripts/estimator-entry.js — mounts the component onto the element your page provides:

import { mount } from "svelte";
import FeeCalculator from "../components/FeeCalculator.svelte";

mount(FeeCalculator, { target: document.getElementById("estimator") });

scripts/build-components.mjs — compiles and bundles to one plain file under site/assets/js/, inside the source tree, where unify finds it like any other asset:

import esbuild from "esbuild";
import sveltePlugin from "esbuild-svelte";

await esbuild.build({
  entryPoints: ["scripts/estimator-entry.js"],
  bundle: true,
  minify: true,
  format: "iife",
  outfile: "site/assets/js/estimator.js",
  plugins: [sveltePlugin()],
});
console.log("built site/assets/js/estimator.js");

On the page that hosts it — say site/courses.html — the script is linked relative to the page's own file, so the page previews when opened straight from the folder, and unify rewrites the address for wherever the page is published (a page one directory down writes ../assets/js/estimator.js):

<div id="estimator"></div>
<script src="assets/js/estimator.js" defer></script>

And the repeatable build is one line:

node scripts/build-components.mjs && unify build

That is the whole integration. bundle: true matters: the compiler's raw output imports Svelte's runtime from node_modules, which a browser cannot resolve and unify does not ship — bundling folds the runtime into the one file. The output is a few tens of kilobytes; it contains no import of anything.

Never hand-translate the component

The one failure mode that matters is not technical, and a build that "works" hides it: rewriting the component by hand in plain JavaScript and keeping the .svelte file as decoration. The site looks identical. The next revision of the component then half-applies or silently doesn't, because nothing actually reads it. The component's own language is the contract: if it is maintained in Svelte, the real Svelte compiler must sit between the .svelte file and the browser.

The test is mechanical, and worth running once after wiring anything up:

  1. Change the component's markup — add a visible line.
  2. Run your build command.
  3. Look for the change in the emitted file: grep "visible line" site/assets/js/estimator.js.

If a value change propagates but a markup change does not, the pipeline is a counterfeit — something is extracting numbers instead of compiling. (In the experiment that produced this document, two of six independent builds were exactly that, one of them importing the real compiler and never calling it.)

The same shape for anything else

TypeScript, JSX, Sass — identical pattern: compiler runs from scripts/ at the project root, output lands in site/assets/ as an ordinary file, unify ships it untouched. unify will never run npm for you, watch your components, or rewrite your bundle: one tool composes HTML, your toolchain makes assets, and the seam between them is the filesystem.

Five recipes

The pattern above runs your toolchain beside unify, by you. The five recipes below are the common variations, and the first of them is the only place unify reaches out at all.

1. The generator context: what --generate hands you

generate: scripts/gen.mjs in unify.yaml — a path in the file counts from the file's own directory, the project root, so it names the scripts/ beside site/ — runs one file you wrote, before unify scans anything. The whole interface is three positional arguments:

const [, , sourceRoot, generatedDir, contextPath] = process.argv;

sourceRoot is the absolute path of your source tree; generatedDir is an absolute path to an empty directory that exists only for this build. Files you write into generatedDir join the build as an overlay — scanned, composed, checked, and published exactly like files in site/. Files you write anywhere else are your own business, and unify neither collects them nor notices them.

contextPath is the absolute path of generator-context.json, a small versioned snapshot unify writes fresh for this one build and deletes when the build finishes, success or failure:

{
  "schemaVersion": 1,
  "unifyVersion": "0.9.0",
  "command": "build",
  "paths": {
    "sourceRoot": "/project/site",
    "generatedRoot": "/tmp/unify-generated-abc123/overlay",
    "outputRoot": "/project/dist"
  },
  "site": {
    "baseUrl": "https://example.com/",
    "prettyUrls": true,
    "canonical": "auto"
  },
  "outputs": {
    "catalog": "assets/unify/catalog.json",
    "searchCorpus": null
  },
  "inputs": {
    "sourcePages": null
  }
}

site and outputs are the flags that change what a generator would otherwise have to duplicate: the effective --base-url/--pretty-urls/--canonical, and the output-relative paths --catalog/--search-corpus will write, each null when its flag is off. inputs.sourcePages is null too unless you ask for the source inventory (below). There is nothing else in it — no settings dump, no environment, no internal option names, and no manifest, because the generator runs before unify has composed a single page. Reading it is optional: sourceRoot and generatedDir are the same two arguments the flag has always passed, unchanged, so a generator that never looks at contextPath keeps working exactly as it did before this third argument existed.

There is nothing to import. A complete generator is this:

import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";

const [, , sourceRoot, generatedDir, contextPath] = process.argv;
const { site } = JSON.parse(readFileSync(contextPath, "utf8"));
mkdirSync(generatedDir, { recursive: true });
writeFileSync(
  join(generatedDir, "credits.html"),
  `<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>Credits</title><meta name="description" content="Who built this."></head>
<body><h1>Credits</h1><p>Built from ${sourceRoot}, published at ${site.baseUrl ?? "no --base-url"}.</p></body>
</html>
`,
);

Five properties are worth knowing before you write a longer one:

An index of your pages, in one build

A generator runs before unify has read a single page, so by default it can only read your files itself. --source-inventory hands it that reading, done: inputs.sourcePages in the context names source-pages.json, one record per source page with the title, description and date its author wrote (null where there is none), the page's source path, and an href you can link to. It is the same list of pages the build treats as pages: _-prefixed files, --exclude matches, fragments and layouts are not in it, a noindex page is. Markdown pages read their frontmatter; HTML pages read their own <head>, as written, with <include>s not resolved. It is source facts only: no layout suffix on titles, no generated pages, no rendered headings.

This generator writes a reports/index.html listing every page under reports/, newest first, so the index and the reports it links to come out of one unify build:

import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";

const [, , , generatedDir, contextPath] = process.argv;
const context = JSON.parse(readFileSync(contextPath, "utf8"));
const { pages } = JSON.parse(readFileSync(context.inputs.sourcePages, "utf8"));

const esc = (s) => s.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/"/g, "&quot;");
const items = pages
  .filter((p) => p.source.startsWith("reports/"))
  .sort((a, b) => (b.date ?? "").localeCompare(a.date ?? ""))
  .map((p) => `<li><a href="${esc(p.href)}">${esc(p.title ?? p.source)}</a>${p.description ? ` — ${esc(p.description)}` : ""}</li>`);

mkdirSync(join(generatedDir, "reports"), { recursive: true });
writeFileSync(
  join(generatedDir, "reports", "index.html"),
  `<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>Reports</title><meta name="description" content="Every report, newest first."></head>
<body><h1>Reports</h1><ul>
${items.join("\n")}
</ul></body>
</html>
`,
);

unify build --generate ../scripts/reports.mjs --source-inventory --audit --strict

On the command line --generate counts from the source root, hence the ../ to reach scripts/ beside site/; in unify.yaml the same file is generate: scripts/reports.mjs, relative to the file. The hrefs are ordinary links to your source pages, so --pretty-urls and a --base-url path prefix rewrite them like links you typed, and the audit checks every one. Put generate: scripts/reports.mjs and source-inventory: true in unify.yaml and the command is just unify build --audit --strict.

Every record also carries meta and links: the <meta> and <link> elements the page itself declares, in order, as attribute records ({"name": "tags", "content": "homelab"}, {"property": "og:image", …}, {"rel": "canonical", "href": …}). A Markdown page's meta is what its frontmatter emits (a list becomes one record per item; title, layout, class, lang and dir are not metas), and its links is always empty. unify gives none of these a meaning: your script does. This one lists articles by series, in part order, keeps each page's tags, and leaves out pages marked role: bookmark:

const metas = (p, name) => p.meta.filter((m) => m.name === name).map((m) => m.content);
const one = (p, name) => metas(p, name)[0] ?? null;

const articles = pages.filter((p) => p.source.startsWith("articles/") && one(p, "role") !== "bookmark");
const bySeries = Map.groupBy(articles, (p) => one(p, "series") ?? "Other");
const sections = [...bySeries].map(([series, list]) => {
  list.sort((a, b) => Number(one(a, "part")) - Number(one(b, "part")));
  const items = list.map((p) => `<li><a href="${esc(p.href)}">${esc(p.title ?? p.source)}</a> ${esc(metas(p, "tags").join(", "))}</li>`);
  return `<h2>${esc(series)}</h2><ol>\n${items.join("\n")}\n</ol>`;
});

It slots into the generator above in place of items; write sections.join("\n") into the page instead.

The blog template ships this worked: unify init blog writes a scripts/gen.mjs beside site/, named by its unify.yaml, that reads posts/*.md and _data/authors.json and writes the index and the feed into the overlay on every build.

2. Image optimization

unify copies every non-page file byte-for-byte, so a 4 MB photograph in your source tree is a 4 MB photograph on your site. unify will not resize it, re-encode it, or generate derivatives — that is a job with real decisions in it (which sizes, which formats, what quality), and a tool that guessed would guess wrong quietly.

Run a real image tool, and run it where its output is cached. The shape that works: originals live in the source tree under an underscore (site/_originals/), so they are build material that never publishes; derivatives land beside the pages that use them.

// scripts/images.mjs — generate: scripts/images.mjs in unify.yaml, or by hand before
// unify build: node scripts/images.mjs site
import { mkdirSync, readdirSync, statSync } from "node:fs";
import { join } from "node:path";
import sharp from "sharp";

const [, , sourceRoot] = process.argv;
const from = join(sourceRoot, "_originals");
const to = join(sourceRoot, "assets/img");
mkdirSync(to, { recursive: true });

for (const name of readdirSync(from)) {
  if (!/\.(jpe?g|png)$/i.test(name)) continue;
  for (const width of [480, 960, 1920]) {
    const out = join(to, `${name.replace(/\.[^.]+$/, "")}-${width}.webp`);
    const src = join(from, name);
    // Skip work already done: this runs on every rebuild.
    try {
      if (statSync(out).mtimeMs >= statSync(src).mtimeMs) continue;
    } catch { /* not built yet */ }
    await sharp(src).resize({ width }).webp({ quality: 80 }).toFile(out);
  }
}

site/_originals/ is excluded by the default _* glob, so the masters never publish; the derivatives sit in site/assets/img/ and ship like any other file. Reference them with an ordinary srcset, which unify rewrites like any other URL:

<img src="/assets/img/anvil-960.webp"
     srcset="/assets/img/anvil-480.webp 480w, /assets/img/anvil-960.webp 960w"
     width="960" height="540" alt="A blacksmith's anvil">

Write width and height on every <img>. Nothing in unify requires it; every browser uses it to reserve space before the image loads, and unify audit will tell you when a page's own share image is missing dimensions.

3. An external CMS over the source tree

Content in a CMS reaches a unify site the same way everything else does: as files in the source tree, written before the build. A generator can fetch and write them, but read the warning first — this is the one recipe where the obvious version is wrong.

A generator runs on every rebuild, and unify dev rebuilds on every save. A generator that calls a CMS API directly makes your editor loop network-dependent, rate-limited, and slow, and it makes your build non-reproducible: the same source tree publishes different sites on different days. Split it in two.

The fetch is a separate command you run when content changes:

// scripts/pull-cms.mjs — run by hand from the project root: node scripts/pull-cms.mjs
import { mkdirSync, writeFileSync } from "node:fs";

const res = await fetch("https://cms.example/api/posts");
if (!res.ok) throw new Error(`CMS returned ${res.status}`);
mkdirSync("site/_cms", { recursive: true });
writeFileSync("site/_cms/posts.json", JSON.stringify(await res.json(), null, 2));

The generator only reads what the fetch left on disk, so it is offline, fast, and deterministic:

// scripts/gen.mjs — generate: scripts/gen.mjs in unify.yaml; runs on every build
import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path";

const [, , sourceRoot, generatedDir] = process.argv;
const posts = JSON.parse(readFileSync(join(sourceRoot, "_cms/posts.json"), "utf8"));
mkdirSync(join(generatedDir, "posts"), { recursive: true });

for (const post of posts) {
  writeFileSync(
    join(generatedDir, "posts", `${post.slug}.md`),
    `---\ntitle: ${JSON.stringify(post.title)}\ndescription: ${JSON.stringify(post.summary)}\ndate: ${post.publishedAt}\nschema: BlogPosting\n---\n\n${post.body}\n`,
  );
}

Commit site/_cms/posts.json. It is excluded from the output by the default _* glob, it makes every build reproducible from the checkout alone, and it turns "the CMS was down" into a problem you have at pull-cms time rather than at deploy time.

Two details that bite. Quote every value you interpolate into frontmatter — JSON.stringify above is doing that job, and a title containing a colon is exactly the value that breaks an unquoted one. And name the fields you emit, one at a time, rather than spreading the CMS record: a {...post} spread is how an author's private email address ends up on a public page.

4. Interoperating with post-build tools

dist/ is an ordinary directory of ordinary files. Anything that reads a directory of HTML works on it with no integration at all — a minifier, an image pipeline, a link checker, rsync, a deploy CLI:

unify build --base-url https://example.com/ && npx some-minifier dist/

Two things make this safer than it looks:

For CI, --format json and --format sarif are mechanical views of the same findings the human report shows:

unify audit --base-url https://example.com/ --format json > findings.json
unify audit --base-url https://example.com/ --format sarif > findings.sarif

Every finding carries a stable identifier and a stable fingerprint, so a CI job can suppress a known one without pattern-matching English. --strict turns findings into a non-zero exit when you want the job to fail on them.

The one flag that touches the network is unify audit --external, which fetches the off-origin URLs your site emits and reports the ones that do not resolve. It is opt-in for a reason: it makes the command's result depend on somebody else's uptime. Ordinary builds and ordinary audits never open a socket.

5. A prebuilt package's browser files

Some packages need no toolchain at all: they ship a bundle a browser can load, and all you need is that file in your output. node_modules/ is on the never-shipped list, so it cannot get there by being where it is — and there is no copy flag to name it with. A generator copies it, which keeps the dependency tracked by your package manager instead of by whoever last dragged files into site/.

Anchor on the package's own package.json and join paths from its directory:

// scripts/vendor.mjs — generate: scripts/vendor.mjs in unify.yaml; runs on every build
import { copyFileSync, mkdirSync, readdirSync } from "node:fs";
import { createRequire } from "node:module";
import { dirname, join } from "node:path";

const generatedDir = process.argv[3];
const require = createRequire(import.meta.url);

// `<pkg>/package.json` is the one subpath an `exports` map almost always keeps, so this
// reaches files the package does not export — which is most of them — and resolves the
// same under both runtimes unify supports.
const packageRoot = (pkg) => dirname(require.resolve(`${pkg}/package.json`));

const md = packageRoot("markdown-it");

const FILES = [[join(md, "dist/markdown-it.min.js"), "vendor/markdown-it.js"]];

// A whole directory, filtered — a language pack or an icon set is usually this shape.
// Name the extension you want: a published `dist/` is mostly things you do not.
for (const name of readdirSync(join(md, "dist")).sort()) {
  if (!name.endsWith(".min.js")) continue;
  FILES.push([join(md, "dist", name), `vendor/min/${name}`]);
}

for (const [from, target] of FILES) {
  const to = join(generatedDir, target);
  mkdirSync(dirname(to), { recursive: true });
  copyFileSync(from, to);
}

Your pages then reference the target paths as ordinary site URLs, because that is what they now are:

<script src="/vendor/markdown-it.js"></script>

Do not resolve the subpath directly. require.resolve("markdown-it/dist/browser/x.js") looks simpler and is the trap: a package's exports map decides which subpaths are nameable, most of the files you want are not on it, and the two runtimes disagree about what to do when they are not — Node raises ERR_PACKAGE_PATH_NOT_EXPORTED while Bun resolves it anyway. A generator written that way builds under the standalone binary and fails under npx @fwdslsh/unify. Resolving package.json and joining sidesteps the map entirely.

Pick a file the browser can actually execute, and check it yourself. Packages ship several builds side by side: a CommonJS one ending in module.exports = …, an ESM entry whose whole body is import x from "../lib/core.js", and somewhere a self-contained bundle. Only the last one works. unify copies bytes and checks references — it cannot tell you that a file you vendored throws module is not defined, or that it imports a sibling you did not copy. Open the file and look for a bundle that names no siblings.

What the build does and does not check. A <link href>, <script src> or other URL attribute naming a file the generator did not write is an unresolved reference and the build refuses to publish, and so is a url(...) inside a vendored stylesheet — which means a CSS file referencing fonts blocks the build until you vendor those too. Nothing inside a .js file is checked, static import and dynamic import() alike, and neither is a bare @import "…" in CSS. So one HTML-referenced file is a tripwire for the whole set, and everything reached from JavaScript is on you.

Four things worth knowing before you run it. There is one generator per build — a second --generate is a usage error, so this code goes inside whatever generator you already have (import it and call it from there). Delete any hand-vendored copies first, or the same output path from both trees is a collision that stops the build. Install before you build: on Node a missing node_modules fails the build with a resolution error, while Bun quietly downloads the package mid-build, so a CI job that skips npm ci either breaks or publishes something nobody pinned. And in --dry-run every vendored file reports its origin as ← generated, which is how you tell a copied dependency from a file you wrote.

What stays outside unify

Plainly, so you can plan around it rather than discover it:

Each of these is a decision rather than a gap. The seam is the filesystem, in both directions, and it stays narrow enough to hold in your head.

For the case where the other tool is a whole static-site generator — Eleventy owning collections, pagination and a data cascade while unify owns every page's composition — see the Eleventy + htmx guide, worked end to end in examples/eleventy-htmx.