Drizzle migrations only run on restore, so a deploy carrying a new migration never applies it #63

New issue

Open

opened 2026-08-19 21:43:09 +00:00 by forgejo-admin · 0 comments

forgejo-admin commented

2026-08-19 21:43:09 +00:00

Owner

Found while designing an app server for byoi (build-your-own-internet), which was going to copy lunacycle's migration approach. It should not copy how the migration is triggered.

What lunacycle does right

scripts/migrate.sh walks Drizzle's meta/_journal.json and applies pending .sql files with sqlite3. It tracks what has been applied by row count in __drizzle_migrations rather than by hash, because Drizzle stores a rolling snapshot hash and not a per-file checksum, so hash matching fails on any database the ORM itself migrated. The script itself is sound, and the comment explaining the row-count decision is good.

It is copied into dist/ by manifest.yml:221,228.

The defect

migrate.sh has exactly one caller in the tree:

celilo/scripts/restore.ts:41   const remoteMigrateScript = '/opt/lunacycle/migrate.sh';

The module's hooks are on_install, on_uninstall, health_check, on_backup, on_backup_analyze, and on_restore. Only the restore path migrates.

So deploying a new lunacycle version that carries a new migration does not apply it. The app comes up against the old schema and stays quiet until something reads or writes a column that is not there.

This is celilo#169 ("apt upgrade does not run DB migrations") live in a module whose state is VERIFIED. modules/wireguard-manager/server/src/store.ts cites celilo#169 as its reason for using plain bun:sqlite with CREATE TABLE IF NOT EXISTS and no migration runner at all, which is a different and also valid answer to the same hazard.

Two ways to fix it

Call migrate.sh from on_install as well as on_restore. Smallest change. It still depends on someone remembering to add the call to any future hook that can land a new version.
Move the migration run into the app at startup. An upgrade restarts the unit, so an upgrade migrates. There is no hook anyone can forget. This is the approach byoi is taking, written up in its design doc as decision D8.

Either way, the gate should be seen to fail before it is trusted: deploy a version carrying a pending migration and confirm the schema actually changed without anyone running a hook by hand.

Smaller thing worth a look while in there

Tracking applied migrations by position rather than by hash means deleting or reordering an entry in _journal.json silently misaligns every migration after it. The comment in migrate.sh explains why hashes were rejected, so this may well be the right trade, but it deserves a line in the script saying so rather than being implicit in the arithmetic.

Not affected

byoi needs no change. Its design already deviates.

Found while designing an app server for byoi (build-your-own-internet), which was going to copy lunacycle's migration approach. It should not copy how the migration is triggered. ## What lunacycle does right `scripts/migrate.sh` walks Drizzle's `meta/_journal.json` and applies pending `.sql` files with `sqlite3`. It tracks what has been applied by row count in `__drizzle_migrations` rather than by hash, because Drizzle stores a rolling snapshot hash and not a per-file checksum, so hash matching fails on any database the ORM itself migrated. The script itself is sound, and the comment explaining the row-count decision is good. It is copied into `dist/` by `manifest.yml:221,228`. ## The defect `migrate.sh` has exactly one caller in the tree: ``` celilo/scripts/restore.ts:41 const remoteMigrateScript = '/opt/lunacycle/migrate.sh'; ``` The module's hooks are `on_install`, `on_uninstall`, `health_check`, `on_backup`, `on_backup_analyze`, and `on_restore`. Only the restore path migrates. So deploying a new lunacycle version that carries a new migration does not apply it. The app comes up against the old schema and stays quiet until something reads or writes a column that is not there. This is celilo#169 ("apt upgrade does not run DB migrations") live in a module whose state is `VERIFIED`. `modules/wireguard-manager/server/src/store.ts` cites celilo#169 as its reason for using plain `bun:sqlite` with `CREATE TABLE IF NOT EXISTS` and no migration runner at all, which is a different and also valid answer to the same hazard. ## Two ways to fix it 1. **Call `migrate.sh` from `on_install` as well as `on_restore`.** Smallest change. It still depends on someone remembering to add the call to any future hook that can land a new version. 2. **Move the migration run into the app at startup.** An upgrade restarts the unit, so an upgrade migrates. There is no hook anyone can forget. This is the approach byoi is taking, written up in its design doc as decision D8. Either way, the gate should be seen to fail before it is trusted: deploy a version carrying a pending migration and confirm the schema actually changed without anyone running a hook by hand. ## Smaller thing worth a look while in there Tracking applied migrations by position rather than by hash means deleting or reordering an entry in `_journal.json` silently misaligns every migration after it. The comment in `migrate.sh` explains why hashes were rejected, so this may well be the right trade, but it deserves a line in the script saying so rather than being implicit in the arithmetic. ## Not affected byoi needs no change. Its design already deviates.