go/migrate/README.md

migrate

go.auvem.com/corestack/migrate sets the house configuration for Goose's Provider API. It returns *goose.Provider directly: Goose owns execution, version tracking, status, and errors. There is no lifecycle dependency or background work.

Use the standalone corestack-goose development tool for SQL and Go creation and timestamp-to-sequential reconciliation. Its generated Go migrations work unchanged with this package's provider-local registration:

go install go.auvem.com/corestack/cmd/corestack-goose@<release>
mkdir -p migrations
corestack-goose create --dir ./migrations create_events sql
corestack-goose fix --dir ./migrations
corestack-goose create --dir ./migrations populate_events go
corestack-goose fix --dir ./migrations
# Or use the version selected by your consumer module:
go run go.auvem.com/corestack/cmd/corestack-goose create --dir ./migrations add_index sql

The directory must exist and contain regular migration files, with no symlink components in its path (on macOS use /private/tmp, rather than /tmp). Names are lowercase ASCII words separated by underscores. Creation uses Goose's stock SQL template and a custom transactional Go template. Go files use the fixed <version>_<name>_migration.go suffix, so names such as test, create_test and windows stay buildable on every platform. Edit the SQL or Go up/down function bodies; leave factories and the generated register.go aggregate intact. GoMigrations() supplies fresh registrations on each call. The aggregate exists even for SQL-only sets. Versions are unique across SQL and Go; Goose timestamps have second resolution, so fix a new migration before creating another in the same second, or wait for a distinct timestamp.

fix preserves sequential IDs, allocates timestamps after the highest sequential ID with at least five-digit padding, and updates filenames and registration literals together. It preserves migration bodies. It rejects unsupported Go layouts, edited aggregates, duplicate versions/factories, unsafe paths, collisions, overflow and lexical/numeric order disagreement before writing. It does not adapt stock Goose Go output, whose global init registry is incompatible here. For concepts, see Goose's hybrid versioning and SQL annotations.

Before fix, commit or separately back up the complete source set, including untracked migrations and tracked edits. Run one authoring process with no concurrent editor changes. New destinations are exclusive; the tool stages output and fully publishes each destination before removing its old source. This is not an atomic multi-file operation. On a write/removal failure, stop and inspect the reported completed, partial/current and outstanding paths. Restore the complete source set and aggregate from that commit/backup before retrying; ordinary Git restore cannot recover untracked files or unsaved edits. Interruption can leave duplicates, aggregate drift or staging files; there is no automatic rollback or journal.

Source reconciliation never rewrites database history. Use it before those IDs reach a shared database; preserve deployed IDs and add migrations afterward. The tool cannot determine deployment history.

The standalone tool has no database setup. Production imports migrate and, optionally, migrate/lifecycle; neither imports authoring code. No build tags are needed. The generated consumer imports only Goose and the standard library. The artifact test builds both executables, decodes production dependency identities, and checks linker symbols for project authoring and Goose Create/CreateWithTemplate/ Fix/SetSequential entrypoints. Goose's package remains a runtime dependency; these checks establish the tested build's boundary, not a universal ban on file I/O or proof about arbitrary downstream binaries. Build your application normally.

Embed migrations in the application's binary and run them before serving traffic:

package app

import (
    "context"
    "database/sql"
    "embed"
    "log/slog"

    "github.com/pressly/goose/v3"
    "go.auvem.com/corestack/migrate"
)

//go:embed migrations/*.sql
var migrations embed.FS

func migrateDatabase(ctx context.Context, db *sql.DB, logger *slog.Logger) error {
    provider, err := migrate.New(db, goose.DialectPostgres, migrations, "migrations", logger)
    if err != nil {
        return err
    }
    _, err = provider.Up(ctx)
    return err
}

The caller opens and closes db, selects a matching database driver and dialect, supplies a nonnil logger, and controls the operation context/deadline. Construction is offline; SQL is parsed lazily by Goose. Provider.Close also closes db, so do not defer it when sharing the pool with the application. Use "." for SQL files at the embedded root. Files in the selected directory are discovered directly, without recursively searching subdirectories.

Preserve the tool's creator/fix filenames and ship the complete migration history. The pinned Goose SQL-only provider relies on lexical filename order matching numeric version order. Timestamp versions and gaps are supported; never derive versions from file counts. For operations, errors, and SQL annotations, use the Provider API and SQL migration docs.

For generated Go functions, import the consumer's migration package and pass migrations.GoMigrations()... as the trailing arguments to migrate.New. Embed the complete directory with //go:embed migrations (and use a different variable name, such as migrationFiles, from the imported package). The directory must exist even for Go-only sets. Global registration is disabled; never share mutable registration objects across providers. See Goose's provider-local registration.

Lifecycle startup and native commands

go.auvem.com/corestack/migrate/lifecycle is a thin adapter; migrate itself has no lifecycle dependency. Alias it migratelifecycle to distinguish the core lifecycle package. Its resolver runs after the caller's database setup and should construct a fresh provider with fresh registrations:

resolve := func(ctx context.Context) (*goose.Provider, error) {
    return migrate.New(db, goose.DialectPostgres, migrationFiles,
        "migrations", logger, migrations.GoMigrations()...)
}
startup, err := migratelifecycle.Module("migrations", migratelifecycle.Require, resolve)
if err != nil {
    return err
}
root := &urfavecli.Command{Name: "app", Commands: []*urfavecli.Command{
    {Name: "serve", Action: cli.Serve(cfg, []lifecycle.Module{database, startup, server})},
    migratelifecycle.Commands(cfg, []lifecycle.Module{database}, resolve),
}}
code, err := cli.Run(ctx, root, args) // args includes argv[0]; caller owns process exit.

Here database.Setup opens db and cleans up its partial acquisition on failure; database.Teardown closes it. Construct the tree and captured resource bindings fresh for independent invocations. Declaration does no work. Select Apply to run Up during setup, or Require to fail before later setup/services when migrations are pending. Require is a point-in-time check without a deployment lock. It and status may initialize Goose's version table; neither promises read-only DB access. The migration module never closes the provider or shared database.

The native group supplies migrate up, up-to <version> (inclusive), down (one migration), down-to <version> and status (version/state/source table). Pass exactly the database modules, excluding the startup migration module and unrelated modules. Services are suppressed; the existing lifecycle/cli.Action owns signals, cancellation, errors and reverse cleanup. Help and invalid arguments never run setup. Target versions must be nonnegative decimal int64 values, with no extra arguments; malformed usage returns exit 2 through cli.Run. Zero follows Goose: down-to 0 rolls everything back; up-to 0 succeeds if nothing is pending, but returns a runtime error when pending work reaches Goose's version validation. Resolver/Goose/cleanup and status Writer failures return runtime errors (exit 1). Supply the native command Writer and configured Goose logger for output.

Postgres providers automatically use Goose's default session advisory locker. Concurrent startup calls from separate providers/processes serialize migration execution and recheck applied versions under the lock. The default lock ID serializes Goose migrations throughout the same database, including different schemas. Use a direct connection or a session-pooling proxy: transaction pooling cannot preserve session locks. RunDB Go migrations need spare pool connections while the locked session is held; avoid a one-connection pool for them.

Other dialects have no cross-provider/process lock configured by this package. Serialize migration runs externally, for example with a single deployment job. SQLite tests validate the SQL convention, not concurrent daemon startup safety.

Run the integration suite against an existing Postgres database with schema creation privileges:

MIGRATE_POSTGRES_DSN='postgres://user:password@localhost/testdb?sslmode=disable' go test -race ./migrate

Tests create and clean only random schemas they own. Without a DSN, Postgres tests skip locally; in CI a missing DSN fails, and the CI job supplies a Postgres service. SQLite tests always run in process using temporary databases.