docs: propose migration authoring and lifecycle integration
This commit is contained in:
end committed
1 parent
0034a561ba
commit
a8e4a0237c
1 file changed
+92
+92
@@ -0,0 +1,92 @@
|
||||
---
|
||||
title: "Separate migration authoring tools from lifecycle runtime adapters"
|
||||
date: 2026-10-28
|
||||
status: proposed
|
||||
implementation: not-started
|
||||
summary: Keep startup and native migration commands in a thin lifecycle adapter and source creation and reconciliation in a standalone development tool.
|
||||
tags:
|
||||
- migrations
|
||||
- lifecycle
|
||||
- cli
|
||||
supersedes: []
|
||||
superseded_by: []
|
||||
related:
|
||||
- /docs/adr/20261025T214616275Z-version-light-core-packages-in-the-root-module-and-nest-heavy-dependency-modules.md
|
||||
- /docs/adr/20261027T184038082Z-supervise-module-lifecycle-and-managed-workers-within-one-shutdown-budget.md
|
||||
- /docs/adr/20261028T210148136Z-run-native-urfave-commands-inside-selected-lifecycle-modules.md
|
||||
tracking:
|
||||
- https://gitea.auvem.com/corestack/go/issues/9
|
||||
implemented_by: []
|
||||
---
|
||||
|
||||
# Separate migration authoring tools from lifecycle runtime adapters
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
[Issue 9](https://gitea.auvem.com/corestack/go/issues/9) requires caller-selected apply/require startup policy and ready-made up, down-to and status commands using only database modules. Its completion gate also requires SQL/Go creation and timestamp-to-sequential reconciliation without manually adapting registrations. The owner chose a standalone `cmd/corestack-goose` tool, available through go install/go run without tags, with its authoring code absent from production applications. Existing `migrate.New` returns a Goose provider with global registration disabled; stock Goose Go output instead registers through init and cannot satisfy this contract.
|
||||
|
||||
The accepted lifecycle and CLI decisions establish ordered setup, reverse cleanup, caller-owned resources, native urfave/cli commands and one shutdown budget. Preserve these boundaries and keep migrate usable without lifecycle. This proposal uses the issue and current source, with no go-toolkit sources. The standalone boundary is confirmed; the API and bounded authoring convention below are proposed defaults requiring independent review and owner acceptance before dependent implementation.
|
||||
|
||||
## Considered Options
|
||||
|
||||
* Let each application wire startup and runtime commands, and manually adapt stock Goose output: least library code, but fails the requested reusable commands and unchanged generated-Go workflow.
|
||||
* Include authoring behind opt-in application build tags: could satisfy the production boundary, but complicates every consumer's build and was declined in favor of a separate tool.
|
||||
* Use a thin runtime adapter plus a standalone authoring tool with a generated registration aggregate: meets the approved artifact boundary and confines source changes to a known convention.
|
||||
* Rewrite arbitrary existing Go registration code during fix, or infer registrations through globals/init: broadens compatibility at substantial parser and state cost; neither is required. Support the house generated convention and reject unsupported layouts before writing.
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
Propose the third option in the root module, with no new dependencies or lifecycle core changes. Runtime lives in `go.auvem.com/corestack/migrate/lifecycle` (package name lifecycle; examples alias it migratelifecycle). It may import migrate's dependencies and `lifecycle/cli`; neither migrate nor lifecycle imports this adapter. Authoring entry points live only in `cmd/corestack-goose` and its internal implementation, never imported by runtime packages or generated consumer code.
|
||||
|
||||
The proposed public runtime surface is:
|
||||
|
||||
```go
|
||||
type Policy string
|
||||
const (
|
||||
Apply Policy = "apply"
|
||||
Require Policy = "require"
|
||||
)
|
||||
func Module(name string, policy Policy,
|
||||
provider func(context.Context) (*goose.Provider, error)) (lifecycle.Module, error)
|
||||
func Commands(config lifecycle.Config, modules []lifecycle.Module,
|
||||
provider func(context.Context) (*goose.Provider, error)) *urfavecli.Command
|
||||
```
|
||||
|
||||
Module validates its name, policy and resolver without invoking the resolver. Its Setup resolves a nonnil provider after earlier database setup, then calls Up for Apply or HasPending for Require. Pending migrations produce a contextual setup error and prevent all later module setups and services; provider errors retain their causes. Zero/unknown policy and nil resolver fail rather than choosing a policy silently. The module has no Run and a no-op Teardown: it owns neither the database nor a separate provider resource. In Goose v3.29.0 Provider.Close closes the supplied database, so the adapter must not call it. The database module owns partial acquisition cleanup and closing, under existing lifecycle rules.
|
||||
|
||||
Ordinary closures supply late-bound resources: a database module's Setup opens db; the provider resolver calls migrate.New with that db, embedded files, dialect, logger and `migrations.GoMigrations()...`. Nothing is resolved while declaring commands or modules. Register the database module before the migration module and dependent services. Each resolver invocation constructs fresh Go registrations; no registry, dependency lookup, injected container, stored context, global mutable state or init is added. Resolution itself acquires no independently owned resource; any resolver that does so must clean up its failure locally.
|
||||
|
||||
Commands returns a fresh native group named migrate with up, down-to and status leaves. Each leaf uses `lifecycle/cli.Action` over exactly the caller-supplied database modules, suppressing their services; it does not include the startup migration module or unrelated application modules. After setup it resolves the provider and invokes the corresponding Goose method. Nil resolver/provider and provider-construction errors fail with ordinary runtime errors. Configuration/module validation, cancellation, reverse cleanup, temporary signal subscriptions and exit classification remain the existing adapter's responsibility. Construct fresh command trees/resource bindings for separate invocations; concurrent mutation/reuse is unsupported.
|
||||
|
||||
Help and invalid leaf arguments must not open the database. Use native ArgValidator before lifecycle entry to enforce no positional arguments for up/status and exactly one decimal int64 version >= 0 for down-to, returning `urfavecli.Exit(err, 2)` for usage errors. Zero means roll back all migrations; reject overflow, negative, missing and extra values. Leave the group Action unset for native help/not-found handling. Status writes a small version/state/source table to the selected command Writer and propagates write errors; up/down-to rely on configured Goose logging. No JSON format or additional flag/config schema is introduced. Existing generated-help parse exceptions remain authoritative.
|
||||
|
||||
Require is a point-in-time pending check, not a transaction spanning subsequent service startup or a lock against concurrent deploys. HasPending does not acquire Goose's session locker. It can initialize Goose's version table; status can do so too. Neither runs migration bodies, but neither promises a read-only connection or zero database writes. Apply and runtime writes retain migrate.New's PostgreSQL session lock and its external-serialization requirement for other dialects. No new background worker, poller, retry policy or lock is added.
|
||||
|
||||
The independent development artifact is installed with `go install go.auvem.com/corestack/cmd/corestack-goose@<release>` or invoked from the consumer module with `go run go.auvem.com/corestack/cmd/corestack-goose ...` using its selected dependency version. Its minimal native CLI is:
|
||||
|
||||
```text
|
||||
corestack-goose create --dir ./migrations <name> sql|go
|
||||
corestack-goose fix --dir ./migrations
|
||||
```
|
||||
|
||||
The directory must already exist. Use package migrations for generated Go files, names restricted to lowercase ASCII words separated by underscores, and reject duplicate generated identifiers before writes. No package option, alternate template option or database connection is needed. SQL uses Goose's supported CreateWithTemplate with its stock SQL template; Go supplies a custom template. Creation runs in a temporary staging directory before publishing validated output. Do not call SetSequential, SetLogger or any global registration helper.
|
||||
|
||||
A generated Go file declares an unexported factory `migrationExample(version int64) *goose.Migration` returning `goose.NewGoMigration(version, &goose.GoFunc{RunTx: upExample}, &goose.GoFunc{RunTx: downExample})`, plus editable transactional up/down bodies. The tool-owned `register.go` exports `GoMigrations() []*goose.Migration` returning a new slice of calls such as `migrationExample(20261028220000)`. Create maintains this aggregate even for a SQL-only set, returning an empty slice when there are no Go migrations. That aggregate is the sole explicit registration-version surface: each literal equals the corresponding filename's numeric prefix. Calling GoMigrations twice produces separate Migration and GoFunc values for provider isolation. Generated consumer files import only Goose and the standard library; applications pass the result to migrate.New. The existing embedded-directory requirement still applies, including for Go-only sets.
|
||||
|
||||
Fix scans SQL and supported Go migration filenames together, preserves sequential IDs, sorts timestamp IDs numerically, and allocates sequential IDs after the highest existing sequential ID, with at least five-digit padding. Follow Goose's timestamp classification (a valid UTC timestamp after the Unix epoch), reject invalid/duplicate/overflowed numeric IDs and any resulting numeric/lexical ordering disagreement. Update filenames and regenerate aggregate literals from the same mapping; keep Go factory names independent of versions. Never replace numbers in migration bodies or parse arbitrary Go registration styles. Use go/parser to validate the owned aggregate's narrow function/call/literal shape and its agreement with the current files, and go/format for new Go output. Require the owned marker and expected structure before replacing an existing register.go; refuse handwritten or drifted registration layouts with an actionable diagnostic. Editable migration bodies remain untouched. Validate generated factory identity/signature and package agreement, without attempting to understand user migration logic.
|
||||
|
||||
Both create and fix preflight the complete planned set before mutation: regular files/directories rather than symlink paths, names, version mappings, generated identifiers, aggregate agreement, destination collisions and output syntax. New destinations are opened exclusively, including renamed migration copies; no unknown destination is overwritten. Existing register.go is intentionally replaced only when recognized as tool-owned and unchanged since preflight. Use one authoring process and no concurrent editor mutations during the operation; these checks do not establish filesystem isolation against racing writers. Fix preserves original migration bytes while changing their paths and the owned aggregate.
|
||||
|
||||
There is no atomic multi-file guarantee. Stage output first, publish a destination completely before removing its old source, and stop at the first write/remove failure with completed and outstanding paths in the diagnostic. Interruption can leave duplicate files, mismatched aggregate versions or temporary files; do not automatically rerun or promise rollback. Work on a clean/recoverable source checkout, inspect the reported paths and restore the affected source set from version control before retrying. Preflight must reject an inconsistent set. This bounded manual recovery avoids a persistent journal or transaction framework. Source renumbering never changes database history: run fix before those IDs reach any shared database; after deployment preserve IDs and add a new migration instead. The tool has no database access with which to prove deployment history.
|
||||
|
||||
### Consequences
|
||||
|
||||
* Runtime persistence remains existing Goose schema/version history writes; new writers are development-only source creation, copies/removals and the owned aggregate replacement. Importing unused runtime code starts nothing. There is no new auth boundary, configuration service, worker, timer or dependency.
|
||||
* Expected implementation is 100–200 runtime lines and 250–400 authoring lines, plus focused tests, examples and consumer docs. Touch new adapter/tool files, migrate's authoring guidance/doc comment and test fixtures; no current callers, release publication configuration or dependency changes are required. Escalate material growth, especially unsupported-source compatibility or automated crash recovery, rather than silently enlarging this PR.
|
||||
* Before delivery, test both startup policies and require blocking later startup; all three native commands against SQLite; setup/teardown and context/error paths; usage/help with no database setup; status output errors. Build a production fixture importing only the runtime adapter and the standalone tool separately; verify the production dependency list and binary symbols exclude this project's authoring package and file-writing entry points. Goose's own upstream authoring functions may exist in module source; prove reachable application code excludes authoring rather than claiming the dependency's source is absent.
|
||||
* The decisive real flow must create SQL and Go migrations, fix timestamp IDs, compile the unedited generated registrations, apply against SQLite and verify versions, pending state and provider isolation. Also cover collisions, unsupported/drifted aggregates, duplicate IDs, mixed IDs, repeated fix and an injected partial-write failure followed by documented source restoration. Build/test both supported artifacts. These are future delivery gates, not checks performed by authoring this ADR.
|
||||
* Consumer docs should recommend this standalone workflow and link Goose's concepts instead of reproducing its manual. Accepting this proposal also accepts the narrow generated layout, single-writer operation and version-control recovery boundary. Independent ADR review and explicit owner acceptance remain required; this proposed record authorizes no production implementation.
|
||||
|
||||
## More Information
|
||||
|
||||
Feasibility is source-inspected, not yet validated end to end. Pinned Goose v3.29.0 [CreateWithTemplate](https://github.com/pressly/goose/blob/v3.29.0/create.go) exposes Version/CamelName template values and exclusively creates its output; it does not return the created path and can leave partial output on template errors, motivating staging. [Fix](https://github.com/pressly/goose/blob/v3.29.0/fix.go) only renames filenames and uses os.Rename, so it cannot maintain our aggregate or provide no-overwrite semantics. Reuse creation facilities, not this insufficient reconciliation operation. [NewGoMigration](https://github.com/pressly/goose/blob/v3.29.0/migration.go) and [provider options](https://github.com/pressly/goose/blob/v3.29.0/provider_options.go) support fresh explicit provider-local registrations. [Provider operations](https://github.com/pressly/goose/blob/v3.29.0/provider.go) and [initialization](https://github.com/pressly/goose/blob/v3.29.0/provider_run.go) establish database ownership and version-table initialization behavior. Native [argument validation](https://github.com/urfave/cli/blob/v3.14.0/command_run.go) runs before Action and supports pre-resource usage failures. These pinned sources were read locally; integration and artifact exclusion remain unverified until implementation.
|
||||
Reference in new issue
Block a user