diff --git a/docs/adr/20261028T215512402Z-separate-migration-authoring-tools-from-lifecycle-runtime-adapters.md b/docs/adr/20261028T215512402Z-separate-migration-authoring-tools-from-lifecycle-runtime-adapters.md index ed4fce6..6477e36 100644 --- a/docs/adr/20261028T215512402Z-separate-migration-authoring-tools-from-lifecycle-runtime-adapters.md +++ b/docs/adr/20261028T215512402Z-separate-migration-authoring-tools-from-lifecycle-runtime-adapters.md @@ -26,7 +26,7 @@ implemented_by: [Issue 9](https://gitea.auvem.com/corestack/go/issues/9) needs caller-selected apply/require startup policy, reusable database-only migration commands, and SQL/Go creation and timestamp-to-sequential reconciliation without manually adapting registrations. The owner selected a standalone `cmd/corestack-goose` tool, available through go install/go run without tags, with authoring code absent from production applications. Existing `migrate.New` disables global registration; stock Goose Go output registers through init and cannot satisfy that contract. -Preserve the accepted lifecycle ordering, reverse cleanup, caller-owned resources, native urfave/cli commands and shutdown budget. Keep migrate usable without lifecycle. This decision uses the issue and current source, with no go-toolkit sources. +Preserve the accepted lifecycle ordering, reverse cleanup, caller-owned resources, native urfave/cli commands and shutdown budget. Keep migrate usable without lifecycle. ## Considered Options @@ -37,9 +37,9 @@ Preserve the accepted lifecycle ordering, reverse cleanup, caller-owned resource ## Decision Outcome -Choose the third option in the root module, with no new dependencies or lifecycle core changes. The owner accepted this decision, including `up-to` and single-step `down`, after independent ADR review. Implementation is published in [PR 18](https://gitea.auvem.com/corestack/go/pulls/18). +Choose the third option in the root module, with no new dependencies or lifecycle core changes. The owner accepted this decision, including `up-to` and single-step `down`, after independent ADR review. See [PR 18](https://gitea.auvem.com/corestack/go/pulls/18) for implementation and review evidence. -Runtime lives in `go.auvem.com/corestack/migrate/lifecycle` (package lifecycle, examples alias migratelifecycle). It may import migrate's dependencies and `lifecycle/cli`; neither migrate nor lifecycle imports it. Authoring lives only in `cmd/corestack-goose` and its internal implementation, never imported by runtime packages or generated consumer code. +Runtime lives in `go.auvem.com/corestack/migrate/lifecycle` (examples alias migratelifecycle), importing migrate's dependencies and `lifecycle/cli`; neither migrate nor lifecycle imports it. Authoring lives only in `cmd/corestack-goose` and its internal implementation, never imported by runtime packages or generated consumer code. The public runtime surface is: @@ -78,15 +78,15 @@ Require is a point-in-time check, not a startup transaction or deployment lock; Install the standalone tool with `go install go.auvem.com/corestack/cmd/corestack-goose@` or use `go run go.auvem.com/corestack/cmd/corestack-goose ...` from the consumer module with its selected dependency version: ```text -corestack-goose create --dir ./migrations sql|go -corestack-goose fix --dir ./migrations +corestack-goose create [--dir ] sql|go +corestack-goose fix [--dir ] ``` -The directory must exist. Generated Go uses package migrations and names restricted to lowercase ASCII words separated by underscores; reject duplicate identifiers before writes. Create stages Goose CreateWithTemplate output using stock SQL/custom Go templates. SQL filenames stay unchanged; Go appends `_migration` to produce `__migration.go`, preventing allowed names such as test, create_test, linux or windows from triggering build filtering. Factory identifiers derive from the logical name, excluding that suffix; create and fix agree. No package/template options, build tags, database access or Goose global setters/registration helpers are added. +Both commands default to the current working directory; `--dir` overrides it. The directory must exist. Generated Go uses package migrations and names restricted to lowercase ASCII words separated by underscores; reject duplicate identifiers before writes. Create stages Goose CreateWithTemplate output using stock SQL/custom Go templates. SQL filenames stay unchanged; Go appends `_migration` to produce `__migration.go`, preventing allowed names such as test, create_test, linux or windows from triggering build filtering. Factory identifiers derive from the logical name, excluding that suffix; create and fix agree. Authoring is offline with a fixed source format and no Goose global registration. -Each generated Go file declares an unexported `migrationExample(version int64) *goose.Migration` factory returning `goose.NewGoMigration(version, &goose.GoFunc{RunTx: upExample}, &goose.GoFunc{RunTx: downExample})`, plus editable transactional bodies. Tool-owned `register.go` exports `GoMigrations() []*goose.Migration`, returning fresh Migration/GoFunc values and a new slice of factory calls with literal versions matching filename prefixes. Maintain this sole explicit registration-version surface even for SQL-only sets (empty slice). Consumer output imports only Goose and the standard library; applications pass registrations to migrate.New. The embedded-directory requirement applies to Go-only sets too. +Each generated Go file declares an unexported factory taking a version and returning a fresh `*goose.Migration` with editable transactional up/down bodies. Tool-owned `register.go` exports `GoMigrations() []*goose.Migration`, returning fresh Migration/GoFunc values and a new slice of factory calls with literal versions matching filename prefixes. Maintain this sole explicit registration-version surface even for SQL-only sets (empty slice). Consumer output imports only Goose and the standard library; applications pass registrations to migrate.New. The embedded-directory requirement applies to Go-only sets too. -Fix scans SQL and house Go filenames together, preserves sequential IDs, sorts timestamp IDs numerically, and allocates after the highest sequential ID with at least five-digit padding. Use Goose's timestamp classification (valid UTC timestamp after Unix epoch); reject invalid/duplicate/overflowed IDs and numeric/lexical ordering disagreement. One mapping drives filename changes and aggregate literals; factory names remain version-independent. Preserve migration bytes and bodies. Validate editable factory identity/signature and package agreement with go/parser as needed, and format new Go output with go/format. Replace register.go only with its owned marker and agreement with current filenames/factories: canonical reconstruction compared after go/format suffices, without another aggregate grammar. Reject handwritten/drifted layouts with actionable diagnostics. +Fix scans SQL and house Go filenames together, preserves sequential IDs, sorts timestamp IDs numerically, and allocates after the highest sequential ID with at least five-digit padding. Use Goose's timestamp classification (valid UTC timestamp after Unix epoch); reject invalid/duplicate/overflowed IDs and numeric/lexical ordering disagreement. One mapping drives filename changes and aggregate literals; factory names remain version-independent. Preserve migration bytes and bodies. Validate factory identity/signature and package agreement. Replace register.go only with its owned marker and canonical agreement with current filenames/factories. Reject handwritten/drifted layouts with actionable diagnostics. Create and fix preflight the complete planned set: regular files/directories rather than symlink paths, names, version mappings, identifiers, aggregate agreement, destination collisions and output syntax. Open new destinations exclusively, including renamed copies. Replace existing register.go only when tool-owned and unchanged since preflight. Require a single authoring process and no concurrent editor mutation; this provides no isolation against racing writers. @@ -94,14 +94,14 @@ There is no atomic multi-file guarantee. Stage first, fully publish each destina ### Consequences -* Runtime persistence remains Goose schema/version history. New writers are development-only source creation, copies/removals and owned aggregate replacement. Unused imports start nothing; no new auth boundary, configuration service or persistent process is added. -* Delivery touches adapter/tool files, migrate authoring guidance/doc comment, focused tests, examples and fixtures; no current callers, publication configuration or dependencies need changes. Assess authoring complexity during implementation; escalate material growth such as arbitrary-source compatibility or automated crash recovery. Consumer docs recommend the standalone flow and link Goose concepts rather than reproduce its manual. -* Validate both startup policies (including require blocking later startup), all five leaves against SQLite, lifecycle/context/error paths, help/usage without database setup, and status writer failures. Cover version validation and Goose zero behavior, UpTo targets and single-step Down. -* Build a runtime-only production fixture and standalone tool separately. Check production dependencies/binary symbols exclude this project's authoring package and file-writing entry points; Goose's upstream authoring source may remain in the dependency. Build/test both artifacts. -* The decisive flow creates SQL/Go, fixes timestamp IDs, compiles unedited registrations (including test, create_test and a host-incompatible platform name), applies to SQLite, and verifies versions, pending state and provider isolation. Also test collisions, unsupported/drifted aggregates, duplicate/mixed IDs, repeated fix, and injected partial-write failure followed by documented source restoration. These checks pass locally. +* Goose retains runtime schema/version history. Development-only source writers create files, copy/remove reconciled sources and replace the owned aggregate. Imports start nothing; no persistent worker or new auth boundary is introduced. +* The fixed source convention avoids arbitrary Go rewriting and automated crash recovery, at the cost of rejecting unsupported layouts and requiring manual source restoration after failures. Consumer guidance lives in [migrate/README.md](/migrate/README.md). +* Delivery checks cover Apply/Require startup ordering; all five leaves against SQLite; lifecycle, context, error and Writer failures; help/usage without database setup; version validation and Goose zero-target behavior. +* Build and test the standalone tool and runtime-only production fixture separately. Production dependencies and binary symbols must exclude this project's authoring package and file-writing entry points; upstream Goose authoring source may remain in the dependency. +* The generated-consumer flow creates SQL/Go, fixes IDs, compiles unedited registrations (including build-sensitive names), applies to SQLite and verifies versions, pending state and provider isolation. Safety tests cover collisions, unsupported/drifted aggregates, duplicate/mixed IDs, repeated fix and injected partial-write failure followed by source restoration. ## More Information -The early real flow built the standalone tool, generated SQL and Go sources, reconciled timestamps, compiled unchanged registrations and applied them to SQLite with fresh-provider isolation. Focused fixtures exercise the production runtime boundary. Pinned Goose v3.29.0 [CreateWithTemplate](https://github.com/pressly/goose/blob/v3.29.0/create.go) provides Version/CamelName template values and exclusive creation, but no returned path and possible partial template-error output, motivating staging. [Fix](https://github.com/pressly/goose/blob/v3.29.0/fix.go) uses os.Rename without aggregate maintenance or no-overwrite guarantees; reuse creation, not that reconciliation. [NewGoMigration](https://github.com/pressly/goose/blob/v3.29.0/migration.go) and [provider options](https://github.com/pressly/goose/blob/v3.29.0/provider_options.go) support fresh provider-local registration. [Provider operations](https://github.com/pressly/goose/blob/v3.29.0/provider.go) and [initialization](https://github.com/pressly/goose/blob/v3.29.0/provider_run.go) establish ownership and version-table behavior. Native [argument validation](https://github.com/urfave/cli/blob/v3.14.0/command_run.go) precedes Action. These sources were read locally; the implementation tests exercise integration and artifact exclusion on the local build. +Pinned Goose v3.29.0 [CreateWithTemplate](https://github.com/pressly/goose/blob/v3.29.0/create.go) supplies templates and exclusive creation but can leave partial output, motivating staging. Its [Fix](https://github.com/pressly/goose/blob/v3.29.0/fix.go) lacks aggregate maintenance and no-overwrite guarantees, so only creation is reused. [NewGoMigration](https://github.com/pressly/goose/blob/v3.29.0/migration.go) and [provider options](https://github.com/pressly/goose/blob/v3.29.0/provider_options.go) support fresh provider-local registration. [Provider operations](https://github.com/pressly/goose/blob/v3.29.0/provider.go) and [initialization](https://github.com/pressly/goose/blob/v3.29.0/provider_run.go) establish ownership and version-table behavior. Native [argument validation](https://github.com/urfave/cli/blob/v3.14.0/command_run.go) precedes Action. -Reconciled against latest `origin/main` at `0034a561ba5eade4ee6fb29436c76e4e34fa3a56` plus PR 18. Independent standard and fresh high implementation reviews of `2727ac83db8f75c3f88e746256f198b0326a145a` found no Blocking/Major issues; Minor R1's Goose zero-target prose/comment correction is committed in `66abdaf`. Required local build, vet, race, pinned lint, formatting, docs and root TODO checks passed on the reviewed implementation; docs, formatting, diff and root TODO checks also passed after the correction. Standalone install/run, generated-consumer SQLite flow and production dependency/symbol checks passed; PostgreSQL integrations skipped without DSNs. Published-head CI remains pending; publication records delivery evidence, not CI success, merge or release. +Reconciled against `origin/main` at `0034a561ba5eade4ee6fb29436c76e4e34fa3a56` plus [PR 18](https://gitea.auvem.com/corestack/go/pulls/18). The PR records independent reviews, passing local build/vet/race/pinned-lint/docs/TODO/format checks, standalone install/run, generated-consumer SQLite flow and production artifact checks. PostgreSQL integrations were skipped locally without DSNs; CI, merge and release status remain in the PR. diff --git a/migrate/README.md b/migrate/README.md index d75afd9..56094e9 100644 --- a/migrate/README.md +++ b/migrate/README.md @@ -12,57 +12,53 @@ unchanged with this package's provider-local registration: ```sh go install go.auvem.com/corestack/cmd/corestack-goose@ mkdir -p migrations -corestack-goose create --dir ./migrations create_events sql -corestack-goose fix --dir ./migrations -corestack-goose create --dir ./migrations populate_events go -corestack-goose fix --dir ./migrations +cd migrations +corestack-goose create create_events sql +corestack-goose fix +corestack-goose create populate_events go +corestack-goose fix # Or use the version selected by your consumer module: -go run go.auvem.com/corestack/cmd/corestack-goose create --dir ./migrations add_index sql +go run go.auvem.com/corestack/cmd/corestack-goose create add_index sql ``` -The directory must exist and contain regular migration files, with no symlink -components in its path (on macOS use `/private/tmp`, rather than `/tmp`). Names -are lowercase ASCII words separated by underscores. Creation uses Goose's stock -SQL template and a custom transactional Go template. Go files use the fixed -`__migration.go` suffix, so names such as `test`, `create_test` and -`windows` stay buildable on every platform. Edit the SQL or Go up/down function bodies; -leave factories and the generated `register.go` aggregate intact. -`GoMigrations()` supplies fresh registrations on each call. The aggregate exists -even for SQL-only sets. Versions are unique across SQL and Go; Goose timestamps -have second resolution, so fix a new migration before creating another in the -same second, or wait for a distinct timestamp. +Both commands default to the current working directory; use `--dir ` to +select another existing directory. Paths must have no symlink components (on +macOS use `/private/tmp`, rather than `/tmp`), and migration files must be regular. +Names are lowercase ASCII words separated by underscores. Templates are stock +Goose SQL or transactional Go. The Go suffix `__migration.go` +keeps names such as `test`, `create_test` and `windows` buildable on every platform. +Edit up/down bodies; leave factories and generated `register.go` intact. +Its `GoMigrations()` returns fresh registrations, even for SQL-only sets. -`fix` preserves sequential IDs, allocates timestamps after the highest sequential -ID with at least five-digit padding, and updates filenames and registration -literals together. It preserves migration bodies. It rejects unsupported Go -layouts, edited aggregates, duplicate versions/factories, unsafe paths, collisions, -overflow and lexical/numeric order disagreement before writing. It does not adapt -stock Goose Go output, whose global `init` registry is incompatible here. +Versions must be unique across SQL and Go. Goose timestamps have second +resolution: fix each new migration before creating another in the same second, +or wait for a distinct timestamp. `fix` preserves sequential IDs and migration +bodies, allocates timestamps after the highest sequential ID with at least +five-digit padding, and updates filenames and registration literals together. +Preflight rejects unsafe paths, collisions, unsupported Go layouts, edited +aggregates, duplicate versions/factories, overflow and lexical/numeric order +disagreement. Stock Goose Go output uses an incompatible global `init` registry. For concepts, see Goose's [hybrid versioning](https://github.com/pressly/goose#hybrid-versioning) and [SQL annotations](https://pressly.github.io/goose/documentation/annotations/). Before `fix`, commit or separately back up the **complete source set**, including -untracked migrations and tracked edits. Run one authoring process with no concurrent -editor changes. New destinations are exclusive; the tool stages output and fully -publishes each destination before removing its old source. This is **not an atomic -multi-file operation**. On a write/removal failure, stop and inspect the reported -completed, partial/current and outstanding paths. Restore the complete source set -and aggregate from that commit/backup before retrying; ordinary Git restore cannot -recover untracked files or unsaved edits. Interruption can leave duplicates, -aggregate drift or staging files; there is no automatic rollback or journal. +untracked migrations and tracked edits. Use one authoring process without concurrent +editor changes. New destinations are exclusive; staged destinations are fully +published before old sources are removed. This is **not an atomic multi-file +operation**: interruption can leave duplicates, aggregate drift or staging files, +with no automatic rollback or journal. On failure inspect the reported completed, +partial/current and outstanding paths, then restore the complete source set and +aggregate from backup before retrying. Ordinary Git restore cannot recover +untracked files or unsaved edits. Source reconciliation never rewrites database history. Use it before those IDs reach a shared database; preserve deployed IDs and add migrations afterward. The tool cannot determine deployment history. -The standalone tool has no database setup. Production imports `migrate` and, -optionally, `migrate/lifecycle`; neither imports authoring code. No build tags are -needed. The generated consumer imports only Goose and the standard library. -The artifact test builds both executables, decodes production dependency identities, -and checks linker symbols for project authoring and Goose Create/CreateWithTemplate/ -Fix/SetSequential entrypoints. Goose's package remains a runtime dependency; -these checks establish the tested build's boundary, not a universal ban on file I/O -or proof about arbitrary downstream binaries. Build your application normally. +The standalone tool needs no database or build tags. Production imports `migrate` +and optionally `migrate/lifecycle`; neither imports authoring code. Generated +migrations import only Goose and the standard library. Artifact tests verify the +runtime dependency and executable boundary for the tested build. Embed migrations in the application's binary and run them before serving traffic: @@ -102,9 +98,8 @@ without recursively searching subdirectories. Preserve the tool's creator/fix filenames and ship the complete migration history. The pinned Goose SQL-only provider relies on lexical filename order matching numeric version order. Timestamp versions and gaps are supported; never derive -versions from file counts. For operations, errors, and SQL annotations, use the -[Provider API](https://pkg.go.dev/github.com/pressly/goose/v3#Provider) and -[SQL migration docs](https://pressly.github.io/goose/documentation/annotations/). +versions from file counts. For operations and errors, use the +[Provider API](https://pkg.go.dev/github.com/pressly/goose/v3#Provider). For generated Go functions, import the consumer's migration package and pass `migrations.GoMigrations()...` as the trailing arguments to `migrate.New`.