105 lines
13 KiB
Markdown
105 lines
13 KiB
Markdown
---
|
|
title: "Separate migration authoring tools from lifecycle runtime adapters"
|
|
date: 2026-10-28
|
|
status: accepted
|
|
implementation: not-started
|
|
summary: Keep startup and native migration commands in a thin lifecycle adapter and source creation and reconciliation in a standalone development tool.
|
|
tags:
|
|
- migrations
|
|
- lifecycle
|
|
- cli
|
|
supersedes: []
|
|
superseded_by: []
|
|
related:
|
|
- /docs/adr/20261025T214616275Z-version-light-core-packages-in-the-root-module-and-nest-heavy-dependency-modules.md
|
|
- /docs/adr/20261027T184038082Z-supervise-module-lifecycle-and-managed-workers-within-one-shutdown-budget.md
|
|
- /docs/adr/20261028T210148136Z-run-native-urfave-commands-inside-selected-lifecycle-modules.md
|
|
tracking:
|
|
- https://gitea.auvem.com/corestack/go/issues/9
|
|
implemented_by: []
|
|
---
|
|
|
|
# Separate migration authoring tools from lifecycle runtime adapters
|
|
|
|
## Context and Problem Statement
|
|
|
|
[Issue 9](https://gitea.auvem.com/corestack/go/issues/9) needs caller-selected apply/require startup policy, reusable database-only migration commands, and SQL/Go creation and timestamp-to-sequential reconciliation without manually adapting registrations. The owner selected a standalone `cmd/corestack-goose` tool, available through go install/go run without tags, with authoring code absent from production applications. Existing `migrate.New` disables global registration; stock Goose Go output registers through init and cannot satisfy that contract.
|
|
|
|
Preserve the accepted lifecycle ordering, reverse cleanup, caller-owned resources, native urfave/cli commands and shutdown budget. Keep migrate usable without lifecycle. This decision uses the issue and current source, with no go-toolkit sources.
|
|
|
|
## Considered Options
|
|
|
|
* Application-specific runtime wiring and manually adapted Goose output: least library code, but misses reusable commands and unchanged generated-Go registration.
|
|
* Application build tags for authoring: complicates consumer builds; declined in favor of a standalone tool.
|
|
* Thin runtime adapter and standalone authoring tool with a generated registration aggregate: meets the artifact boundary with a bounded source convention.
|
|
* Rewrite arbitrary Go registrations or infer them through globals/init: broader compatibility and parser/state cost than required; reject unsupported layouts instead.
|
|
|
|
## Decision Outcome
|
|
|
|
Choose the third option in the root module, with no new dependencies or lifecycle core changes. The owner accepted this decision, including `up-to` and single-step `down`. Implementation has not started; independent delta review precedes production code.
|
|
|
|
Runtime lives in `go.auvem.com/corestack/migrate/lifecycle` (package lifecycle, examples alias migratelifecycle). It may import migrate's dependencies and `lifecycle/cli`; neither migrate nor lifecycle imports it. Authoring lives only in `cmd/corestack-goose` and its internal implementation, never imported by runtime packages or generated consumer code.
|
|
|
|
The public runtime surface is:
|
|
|
|
```go
|
|
type Policy string
|
|
const (
|
|
Apply Policy = "apply"
|
|
Require Policy = "require"
|
|
)
|
|
func Module(name string, policy Policy,
|
|
provider func(context.Context) (*goose.Provider, error)) (lifecycle.Module, error)
|
|
func Commands(config lifecycle.Config, modules []lifecycle.Module,
|
|
provider func(context.Context) (*goose.Provider, error)) *urfavecli.Command
|
|
```
|
|
|
|
Module validates name, policy and nonnil resolver without resolving. Setup resolves a nonnil provider after earlier database setup, then calls Up for Apply or HasPending for Require. Pending migrations fail setup before later modules/services; errors retain contextual causes. Zero/unknown policy fails. The module has no Run and no-op Teardown: the database module owns partial acquisition cleanup and closing. Never call Provider.Close, which closes the supplied database in Goose v3.29.0.
|
|
|
|
Ordinary closures bind resources: database Setup opens db; the resolver calls migrate.New with db, embedded files, dialect, logger and `migrations.GoMigrations()...`. Register database before migration and dependent services. Declaration starts nothing. Each resolution constructs fresh Go registrations, without a registry, stored context, global mutable state or init. Resolution acquires no independently owned resource; a resolver that does must clean up its failure locally.
|
|
|
|
Commands returns a fresh native `migrate` group with these leaves:
|
|
|
|
| Leaf | Goose provider method |
|
|
| --- | --- |
|
|
| `up` | Up |
|
|
| `up-to <version>` | UpTo |
|
|
| `down` | Down (one migration) |
|
|
| `down-to <version>` | DownTo |
|
|
| `status` | Status |
|
|
|
|
Each uses `lifecycle/cli.Action` over exactly the supplied database modules, suppressing services and excluding the startup migration module and unrelated modules. After setup, resolve the provider and invoke Goose directly. Nil resolver/provider and construction failures are runtime errors. Existing adapter validation, cancellation, reverse cleanup, signals and exit classification apply. Use fresh trees/resource bindings for independent invocations; concurrent mutation/reuse is unsupported.
|
|
|
|
Help and invalid arguments must not open the database. Native ArgValidator enforces no positional arguments for up/down/status and exactly one nonnegative decimal int64 for up-to/down-to; reject negative, overflow, missing and extra values with `urfavecli.Exit(err, 2)` before lifecycle entry. Zero behavior follows Goose, including DownTo(0) rolling back all migrations. Leave the group Action unset for native help/not-found behavior; existing generated-help exceptions apply. Status writes a version/state/source table to the selected command Writer and propagates write errors. Other leaves use configured Goose logging; no extra output/config schema is added.
|
|
|
|
Require is a point-in-time check, not a startup transaction or deployment lock; HasPending takes no Goose session lock. Require and status can initialize the version table: neither runs migration bodies nor promises read-only database access. Apply and runtime writes retain migrate.New's PostgreSQL session lock and external serialization for other dialects. No new worker, retry or lock is introduced.
|
|
|
|
Install the standalone tool with `go install go.auvem.com/corestack/cmd/corestack-goose@<release>` or use `go run go.auvem.com/corestack/cmd/corestack-goose ...` from the consumer module with its selected dependency version:
|
|
|
|
```text
|
|
corestack-goose create --dir ./migrations <name> sql|go
|
|
corestack-goose fix --dir ./migrations
|
|
```
|
|
|
|
The directory must exist. Generated Go uses package migrations and names restricted to lowercase ASCII words separated by underscores; reject duplicate identifiers before writes. Create stages Goose CreateWithTemplate output using stock SQL/custom Go templates. SQL filenames stay unchanged; Go appends `_migration` to produce `<version>_<name>_migration.go`, preventing allowed names such as test, create_test, linux or windows from triggering build filtering. Factory identifiers derive from the logical name, excluding that suffix; create and fix agree. No package/template options, build tags, database access or Goose global setters/registration helpers are added.
|
|
|
|
Each generated Go file declares an unexported `migrationExample(version int64) *goose.Migration` factory returning `goose.NewGoMigration(version, &goose.GoFunc{RunTx: upExample}, &goose.GoFunc{RunTx: downExample})`, plus editable transactional bodies. Tool-owned `register.go` exports `GoMigrations() []*goose.Migration`, returning fresh Migration/GoFunc values and a new slice of factory calls with literal versions matching filename prefixes. Maintain this sole explicit registration-version surface even for SQL-only sets (empty slice). Consumer output imports only Goose and the standard library; applications pass registrations to migrate.New. The embedded-directory requirement applies to Go-only sets too.
|
|
|
|
Fix scans SQL and house Go filenames together, preserves sequential IDs, sorts timestamp IDs numerically, and allocates after the highest sequential ID with at least five-digit padding. Use Goose's timestamp classification (valid UTC timestamp after Unix epoch); reject invalid/duplicate/overflowed IDs and numeric/lexical ordering disagreement. One mapping drives filename changes and aggregate literals; factory names remain version-independent. Preserve migration bytes and bodies. Validate editable factory identity/signature and package agreement with go/parser as needed, and format new Go output with go/format. Replace register.go only with its owned marker and agreement with current filenames/factories: canonical reconstruction compared after go/format suffices, without another aggregate grammar. Reject handwritten/drifted layouts with actionable diagnostics.
|
|
|
|
Create and fix preflight the complete planned set: regular files/directories rather than symlink paths, names, version mappings, identifiers, aggregate agreement, destination collisions and output syntax. Open new destinations exclusively, including renamed copies. Replace existing register.go only when tool-owned and unchanged since preflight. Require a single authoring process and no concurrent editor mutation; this provides no isolation against racing writers.
|
|
|
|
There is no atomic multi-file guarantee. Stage first, fully publish each destination before removing its source, and stop on the first write/remove failure, reporting completed/outstanding paths. Interruption may leave duplicates, aggregate mismatches or temporary files; no automatic retry or rollback. Before fix, commit or separately back up the complete source set, including untracked migrations and tracked edits; ordinary restore cannot recover untracked files or unsaved changes. On failure inspect reported paths and restore from that commit/backup before retrying; preflight rejects inconsistent sets. This manual recovery needs no persistent journal. Fix changes source IDs, never database history: use it before IDs reach a shared database; preserve deployed IDs and add migrations afterward. The tool cannot prove deployment history.
|
|
|
|
### Consequences
|
|
|
|
* Runtime persistence remains Goose schema/version history. New writers are development-only source creation, copies/removals and owned aggregate replacement. Unused imports start nothing; no new auth boundary, configuration service or persistent process is added.
|
|
* Delivery touches adapter/tool files, migrate authoring guidance/doc comment, focused tests, examples and fixtures; no current callers, publication configuration or dependencies need changes. Assess authoring complexity during implementation; escalate material growth such as arbitrary-source compatibility or automated crash recovery. Consumer docs recommend the standalone flow and link Goose concepts rather than reproduce its manual.
|
|
* Validate both startup policies (including require blocking later startup), all five leaves against SQLite, lifecycle/context/error paths, help/usage without database setup, and status writer failures. Cover version validation and Goose zero behavior, UpTo targets and single-step Down.
|
|
* Build a runtime-only production fixture and standalone tool separately. Check production dependencies/binary symbols exclude this project's authoring package and file-writing entry points; Goose's upstream authoring source may remain in the dependency. Build/test both artifacts.
|
|
* The decisive flow creates SQL/Go, fixes timestamp IDs, compiles unedited registrations (including test, create_test and a host-incompatible platform name), applies to SQLite, and verifies versions, pending state and provider isolation. Also test collisions, unsupported/drifted aggregates, duplicate/mixed IDs, repeated fix, and injected partial-write failure followed by documented source restoration. These remain delivery gates, not completed checks.
|
|
|
|
## More Information
|
|
|
|
Feasibility is source-inspected, not validated end to end. Pinned Goose v3.29.0 [CreateWithTemplate](https://github.com/pressly/goose/blob/v3.29.0/create.go) provides Version/CamelName template values and exclusive creation, but no returned path and possible partial template-error output, motivating staging. [Fix](https://github.com/pressly/goose/blob/v3.29.0/fix.go) uses os.Rename without aggregate maintenance or no-overwrite guarantees; reuse creation, not that reconciliation. [NewGoMigration](https://github.com/pressly/goose/blob/v3.29.0/migration.go) and [provider options](https://github.com/pressly/goose/blob/v3.29.0/provider_options.go) support fresh provider-local registration. [Provider operations](https://github.com/pressly/goose/blob/v3.29.0/provider.go) and [initialization](https://github.com/pressly/goose/blob/v3.29.0/provider_run.go) establish ownership and version-table behavior. Native [argument validation](https://github.com/urfave/cli/blob/v3.14.0/command_run.go) precedes Action. These sources were read locally; integration and artifact exclusion await implementation.
|