Ship the map as a celilo module, with CI/CD and an e2e suite #3
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/celilo-module"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
The map is a blob pasted into Squarespace, so it is only as fresh as the last time somebody pasted it. In 2025 two copies drifted apart with no way to tell which was live.
Solution
Phase one of
add-celilo-hosting, plus the build bus and a test suite.The module was rebuilt against the framework's own guide
The first version of this module was written by copying
celilo-website. There is a 3305-linereference/MODULE_DEVELOPMENT_GUIDE.mdand an@celilo/mcp-dev-serverwith a scaffolder and the authoritative capability and hook contracts. Using them found six defects:requires: dns_registrarwas never this module's dependency. Thepublic_webcontract says it outright: it owns public reachability for the routes it registers and reaches the registrar itself. Copied from celilo-website.public_webwas declared optional with a hand-rolled throw. It is a hardrequiresnow, so the framework refuses before the hook runs.on_install.ts,health_check.ts).on_installhad no tests; it has five.celilo/types.d.tsclaimed to be generated and was not.scripts/tsconfig.jsonreached into a siblingcelilocheckout, so cloning this repo alone gave a broken typecheck.CI/CD: a merge produces a .netapp on the registry
on_installcallsregisterWithForge, which mints a publish token scoped to this one package viaregistry_publishand hands it tosource_forge.registerRepo. That sets the repo'sCELILO_PUBLISH_TOKENsecret and applies branch protection. Idempotent, so every deploy reconciles it.source_forgeandregistry_publishareoptional, notrequires. lunacycle's manifest carries the reason and it applies exactly here: underrequiresthey are enforced at import time, somodule importfails on any fleet without a forge — which is every fleet the e2e builds. The e2e proves this: it imports and deploys with no forge and no registry present.release.ymlre-runs the PR gates on main before publishing, because pr-validate proves each branch is green and cannot prove main is. After publishing it asks the registry what it actually holds and fails if the manifest version is not there — a publish that reports success and ships nothing is a real outcome the exit code does not distinguish.Both workflows run the CLI this repo pins, not
bunx @celilo/cli. That is not tidiness. The CLI version silently decides whether the artifact is usable at all: 2.2.0 strippedscripts/node_modulesout of any module with a rootpackage.json, so it built a.netappthat packaged, published, imported and deployed and then died inon_install(celilo#1310, fixed in 2.2.1).bunxresolveslatestat run time, so a future regression would ship the same way. CI already runsbun install --frozen-lockfile, so the lockfile is the single source of truth for which packager built the artifact, with no version literal duplicated into the workflows.@celilo/cliand@celilo/e2eare pinned exactly rather than caret ranges for the same reason.The pipeline ends at a .netapp in the registry. Nothing in it deploys. That needs
auto_upgradeandupgrade_policy, which are operator state and deliberately not in this repo.Why this repo moved to the celilo org
The fleet runner is scoped
org/celiloand advertisesnative:host. This repo was atforgejo-admin/bna-yard-sale, a user namespace. Forgejo scopes are a tree where org and user are siblings, so the runner could never claim these jobs: every run sat inwaitingforever, with no error anywhere, even though the runner existed and advertised exactly the requested label. That is celilo#627.manifest.test.tsnow pinsforge_repoto the org so it cannot regress quietly.Tests
pipeline.integration.test.ts— real HTTP servers on localhost, not a mocked fetch, because the bugs worth catching here live in the wire format. It serves the exact header Google publishes for this form, where the address question wraps onto a second line inside quotes. The case that matters most: an unpublished sheet answers 200 with a sign-in page, which without a guard parses into one very strange sale. Verified that guard fails when removed.tests/bna-yard-sale.e2e.test.ts— a real cele2e deploy, ten stages. Beyond serving a 200 it fetchessales.jsonand asserts every pin lands inside the neighborhood bounding box; a geocoder misfire that shipped would put a marker on a stranger's house. It also asserts the served page renders seller text as text, and asks the module's own health check for an independent second reading.e2e does not run in CI. The rig is a single machine-global resource behind a run-lock; it is
bun run test:e2elocally, andcele2e build-infrahas to have been run once on the machine to populate the harness's netapp cache.The e2e suite is green on the released toolchain
Ten stages, 142s, against
@celilo/cli2.2.1 and@celilo/e2e0.20.1 from npm — nothing unreleased anywhere in the chain.site verified: 33 sales served/served page renders seller text as text/module reports healthy.Two of the defects it found were ours:
public_web3.2.0, a version nothing has ever provided. The fleet's caddy provides 3.1.0, the current contract. This would have blocked the production deploy too, and no local gate can see it —celilo module checkreads the manifest, not the fleet. The deploy is what fails.bna-yard-sale.org. cele2e's simulated internet is authoritative only for the zones its DNS containers are seeded with, so the fake registrar answeredENOENT ... /config/bna-yard-sale.org.zone. It now usesiamtheinternet.org, which also proves the module takes its hostname from config, since the manifest's default is the real domain and this is not it.Four were in the framework, filed as celilo#1309 and celilo#1310. celilo#1310 is fixed and released. The three in #1309 are still present in 0.20.1, so the suite works around each:
publishModuleshells out to a CLI path that exists only in the celilo monorepo, so packaging fails for every npm consumer. That killsregisterControlPlane(), which every suite calls. We package with the pinned@celilo/cliand hand it a.netapp..netappform takes the id from the filename instead.copyFileSync(path, path)and destroys it — the harness'snetapps/cache went from 39 files to 38.stagedNetappCopycopies into a temp dir first.registerControlPlanedefaults to amodules/celilo-mgmtthat a consumer install ships empty, so it is handed the staged netapp explicitly.assertStandardModulesStagedturns the resulting failure — which otherwise arrives four stages later asModule 'iptables' not found in registryand reads as a broken registry — into the one instruction that fixes it. It has already earned its keep once: a dependency reinstall wiped the netapp cache, and the suite failed in 0s with the remedy instead of 90s into a rig run.Verified on the running system, not just by assertions
Inspected a live deploy with
--keep:sales.jsonis byte-identical to the build output and matches the committed source at 33 entries (closes task 5.5 ofadd-celilo-hosting).@celilo/capabilities,drizzle-orm,tldts,tldts-core.on_installis genuinely idempotent. Three publishes of identical content produced one content-addressed release directory with an unchanged timestamp, so redeploys do not accumulate on the host.sales.jsonis servedcache-control: no-cache. This matters: the capability's Caddy template gives/assets/*a one-year immutable cache, and a data file in that bucket would have frozen the sale list.public_webrenderstry_files {path} {path}/index.html /index.html, so every path returns 200 —/does-not-existserves the map. That comes from the capability, not this module. For a one-page map it is arguably better than a 404, so it is recorded rather than changed.Verification
bun run typecheckbun run typecheck:hooksbun run typecheck:e2ebun run test:unitbun run build:sitecelilo module check .bun run test:e2eThe built site was served and driven in real Chrome: 33 sales fetched, 33 rows, 33 flowers, 18 street labels, map fills its pane, all three derived date strings correct.
What this does not cover
The forge registration path never executes in the e2e, because no e2e fleet has a forge —
registerWithForgewarns and returns. On the production fleet both capabilities exist, so the token-minting code runs for the first time on the real deploy. The logic is unit-tested against the capability contract with fakes (register-forge.test.ts), so what is untested is the real capability implementation, not the branch. Covering it properly would mean deployingforgejo, roughly tripling the suite.Still blocked on one thing
CELILO_PUBLISH_TOKENdoes not exist, so merging this runsrelease.ymland fails at the publish step. Two ways to create it:registerWithForgemints it as designed. Gated on confirmingnamecheapmanagesbna-yard-sale.org..netappfrom the domain question.Nothing in this PR has touched the fleet.
🤖 Generated with Claude Code
https://claude.ai/code/session_018Qbb67pVLEyTPKMo9T8Pee
I built this module by copying celilo-website. There is a 3305-line reference/MODULE_DEVELOPMENT_GUIDE.md and an @celilo/mcp-dev-server with a scaffolder, a manifest validator, and the authoritative capability and hook contracts. I used none of them, because I searched docs/ for "module" and never looked in reference/, and never asked whether tooling existed. Scaffolding a module properly and reading describe_capability('public_web') turned up six things. requires no longer lists dns_registrar. The public_web contract says it outright: public_web owns public reachability for the routes it registers and reaches the registrar itself, so "you do NOT requires: dns_registrar just to reach clients". That was copied from celilo-website and was never this module's dependency. public_web is also pinned at the current 3.2.0 rather than 3.0.0. public_web is now a hard `requires` on the hook rather than `optional` with a hand-rolled throw. Optional is the shape for a capability the module can work without; with no public_web there is nowhere to publish, so the framework should refuse before the hook runs. It also makes the capability non-nullable at the type level instead of guarded at runtime. Hook scripts are named for their hooks: on_install.ts and health_check.ts. on_install has unit tests for the first time — five, covering the domain fallback, the missing-capability path, a failed publish, and that the request carries no sourceDir, which the capability removed in D10. tests/bna-yard-sale.e2e.test.ts is the cele2e stub the convention calls for, skipped until @celilo/e2e is consumable from outside the monorepo. Its assertions are written against the artifact: the page answers AND sales.json holds sales the map can draw. celilo/types.d.ts stops claiming to be generated. It carried a "Generated by celilo module types generate — do not edit" banner it had not earned. It now says it is hand-written and mirrors the manifest. scripts/tsconfig.json no longer extends a path inside a sibling celilo checkout. That only resolved because celilo happens to sit next to this repo; anyone cloning this alone got a broken typecheck. It is standalone now. Version bumped 0.1.0 -> 0.2.0: new manifest surface and changed hook contracts, backward-compatible, which the guide's versioning section makes a minor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Qbb67pVLEyTPKMo9T8PeeShip the map as a celilo moduleto Ship the map as a celilo module, with CI/CD and an e2e suite