diff --git a/docs/adr/20261027T184351001Z-preserve-typed-sort-values-in-versioned-page-cursors.md b/docs/adr/20261027T184351001Z-preserve-typed-sort-values-in-versioned-page-cursors.md new file mode 100644 index 0000000..fdd52be --- /dev/null +++ b/docs/adr/20261027T184351001Z-preserve-typed-sort-values-in-versioned-page-cursors.md @@ -0,0 +1,88 @@ +--- +title: Preserve supported value types in versioned page cursors +date: 2026-10-27 +status: proposed +implementation: not-started +summary: Pair a plain sort with a typed row key function and preserve supported concrete values with tagged cursors. +tags: + - pagination + - cursors +supersedes: [] +superseded_by: [] +related: + - /docs/adr/20261025T214616275Z-version-light-core-packages-in-the-root-module-and-nest-heavy-dependency-modules.md +tracking: + - https://gitea.auvem.com/corestack/go/issues/6 +implemented_by: [] +--- + +# Preserve supported value types in versioned page cursors + +## Context and Problem Statement + +[Issue 6](https://gitea.auvem.com/corestack/go/issues/6) requires transport-neutral, non-null keyset pagination through Bob query mods. Cursors may be retained by callers, so their format is a compatibility contract. The owner requires two plain query inputs: a Sort and a typed `func(T) []any` returning column values in sort order. No reflection, struct tags, field lookup or per-column getters may connect rows to columns. Future code generation may produce both values; v0 must work with handwritten values. + +Type preservation here means retaining each supported concrete value across encoding/decoding and passing that value to Bob's argument binder. It does not mean statically proving the agreement between independent Sort and key values. JSON decoded into untagged `any` would lose integer precision and concrete types. + +## Considered Options + +* Plain Sort/key values with explicit tagged scalar encoding: meets the requested API and preserves a documented finite set without reflection. +* Untagged JSON decoded into `[]any`: smaller, but numbers become float64 and concrete type information disappears. +* Per-column generic extractors or caller-defined generic cursor tuples: can tie values to columns more strongly, but do not match the owner's chosen two-value API. +* Reflection, codec registration or arbitrary serialized Go objects: broadens the supported set at the cost of machinery and compatibility obligations not needed for v0. + +## Decision Outcome + +Propose plain Sort/key inputs with tagged JSON, wrapped in unpadded base64url (`base64.RawURLEncoding`). This record remains proposed pending independent review and owner acceptance; implementation has not started. + +Sort contains a stable caller-owned ID and an ordered list of columns, each with a stable ID, Bob expression, direction and unique marker. Reject empty sorts, missing/duplicate IDs, nil expressions, invalid directions and a missing unique marker on the final column. The marker asserts that column alone is unique in the selected relation under database collation/comparison semantics, including joins. The library cannot prove schema truth. Composite-only uniqueness is outside the issue's final unique-column contract. Expressions and IDs come from trusted application declarations, never cursor-controlled SQL. Changing expression, key type or comparison meaning requires a new sort ID. + +A typed key function returns values in exactly that order. Check `len(key(row)) == len(sort.Columns)` for every returned row used to assemble a page, including the fetched extra row. A mismatch returns a clear programmer-error `KeyError` with expected/actual lengths; never truncate, zip a prefix or emit a bad cursor. Reject nil key functions. Column/value correspondence remains an explicit caller contract; no field discovery or hidden mapping is introduced. + +Cursor v1 uses a strict envelope with version, sort ID, ordered column IDs/directions/unique markers, and tagged values. Example decoded JSON: + +```json +{"v":1,"sort":"orders/recent/v1","columns":[{"id":"created","direction":"desc","unique":false},{"id":"id","direction":"asc","unique":true}],"values":[{"type":"time","value":"2026-10-27T12:00:00Z"},{"type":"int64","value":"9007199254740993"}]} +``` + +All payloads are strings. Fixed type tags distinguish `int`, `int8`, `int16`, `int32`, `int64`, `string`, `time` and `uuid`. A type switch encodes exactly these concrete Go types; a fixed switch reconstructs them using range-checked integer parsing, valid UTF-8 strings, `time.Time` via RFC3339Nano, and standard-library `uuid.UUID` via canonical lowercase dashed text. Integer decimal text is canonical and never passes through float64. `int` must fit the consuming platform. Timestamp round trips preserve SQL-relevant instant/precision, not monotonic metadata or named timezone identity. Reject invalid UTF-8 rather than silently replacing bytes. + +UUID support specifically means Go 1.27.1's standard `uuid.UUID`: its Parse/String methods support this representation and database/sql's default converter converts it to a string for drivers. Third-party UUID types require explicit caller conversion to this type. Named user-defined scalar types, unsigned integers, floats, booleans, pointers, nullable wrappers, byte slices and arbitrary custom types return a typed `ValueError`; no reflection, implicit coercion, custom binder or registry is proposed. A true Go type alias retains its underlying supported identity. Additional types require an explicit format/support decision. V0 sort columns must be non-null and ordered consistently by WHERE and ORDER BY; callers extract actual stored/scanned values, including timestamp precision. + +Decode and bound the token before constructing a mod (proposed maximum: 16 KiB encoded). Reject bad base64/JSON, unknown/duplicate/missing fields, trailing data, version mismatch, sort descriptor mismatch, wrong value count, nulls, unknown tags and invalid/range-overflowing payloads. An exported `CursorError`, inspectable with `errors.As`, distinguishes malformed, version, sort and unsupported-value failures. Encoding unsupported key values returns `ValueError` identifying the position and type, without leaking values. Keys and token shape are validated before producing a Page; errors return no partial page. Cursors are not authenticated: a well-formed edit to a supported value or tag remains a seek request, not a tamper-detection failure. Without expected type metadata in Sort, the library cannot prove an edited supported tag matches a SQL column; driver/database errors propagate normally. Authorization stays in the query. + +Prepared pagination exposes one generic Bob `Mod[Q]` targeting native `AppendWhere(...any)`, `AppendOrder(bob.Expression)` and `SetLimit(any)` methods shared by PostgreSQL, SQLite and MySQL SelectQuery. Use Bob expressions and `clause.OrderDef`, fixed operators/directions, and an integer limit. Prepare validates requests/cursors first because ordinary Bob Apply has no error return. Cursor values only enter `expr.Arg(value)`; Bob's existing `any` boundary retains their concrete dynamic type. No custom SQL renderer is needed. Apply once to a query without ORDER BY, LIMIT, OFFSET or FETCH; callers may append their dynamic/authorization filters but must not override pagination/order. + +For `(created DESC, id ASC)`, after means `created < c OR (created = c AND id > i)`, grouped as one WHERE condition. Before reverses each comparison. Backward also reverses each ORDER BY direction, fetches limit+1, trims overflow, then reverses retained rows into canonical order. Page[T] holds Items, StartCursor, EndCursor, HasNext and HasPrevious; cursors describe the canonical first/last returned items, or are empty when no items exist. Deleted anchors remain value boundaries. Writes or changed filters between requests may skip/repeat items; no snapshot is promised. + +Forward maps first/after and Backward maps last/before to a transport-neutral Direction, Size and Cursor request. Proposed defaults: zero direction is Forward, zero/absent size uses configured default, negative sizes and invalid directions fail, oversized sizes clamp to maximum; require `1 <= default <= maximum` and overflow-safe limit+1. Empty cursor means no anchor. The owner selected continuation semantics: forward HasNext=overflow and HasPrevious=nonempty after; backward HasPrevious=overflow and HasNext=nonempty before. This applies even to empty/stale results. An anchor-derived opposite flag does not prove rows exist; no extra existence probes are required. + +A concrete proposed call shape is: + +```go +sort := page.Sort{ID: "orders/recent/v1", Columns: []page.Column{ + {ID: "created", Expression: expr.Quote("orders", "created_at"), Direction: page.Desc}, + {ID: "id", Expression: expr.Quote("orders", "id"), Direction: page.Asc, Unique: true}, +}} +key := func(o Order) []any { return []any{o.CreatedAt, o.ID} } // time.Time, int64 +prepared, err := page.Prepare(sort, key, page.Request{ + Direction: page.Forward, Size: 25, Cursor: after, +}, page.Limits{Default: 25, Maximum: 100}) +// Handle err before applying the mod to a normal Bob PostgreSQL SELECT q. +q.Apply(page.Mod[*pgdialect.SelectQuery](prepared)) +q.Apply(sm.Where(psql.Quote("orders", "tenant_id").EQ(psql.Arg(tenantID)))) +// Execute q with Bob's normal Order mapper, obtaining up to 26 []Order rows. +result, err := prepared.Page(rows) +``` + +Backward changes Direction and supplies the before cursor. `pgdialect`/`sm` are Bob's PostgreSQL dialect/select-mod packages; corresponding SQLite/MySQL select types use the same mod. `Page` retains the typed `Order` key function; `o.ID` is restored as int64, including above 2^53. A separate opt-in `Count(ctx, executor, callerBuiltCountQuery)` executes an explicitly counting, authorized unpaginated Bob query. It never guesses how to strip pagination or count grouped/distinct caller relations. + +Bob v0.50.0 is source-feasible for all three dialects, with two reported limitations: generic `mods.OrderBy` expects a different AppendOrder signature from actual selects (native ModFunc matches); MySQL's select writer discards LIMIT expression arguments (use the normal native integer LIMIT API, while cursor WHERE values remain bound). Runtime dialect execution is unverified. Bob belongs in the root module under the accepted module-boundary ADR; no UUID dependency is needed. No persistence, writers, migrations, secrets, workers or timers are introduced. + +### Consequences + +* The owner-chosen Sort/key separation is simple and code-generation-ready, with runtime length errors instead of compile-time column/value agreement. Tests and package examples must demonstrate correct pairing and disagreement failures. +* Explicit tags retain supported concrete types without reflection, precision loss or a registry. Unsupported custom/third-party types require explicit caller conversion or a later support decision. +* Version and sort IDs become compatibility commitments; changes may deliberately reject retained cursors. No legacy decoder is needed before another version exists. +* Cursors expose values and permit valid edits; queries, counts and every request must preserve authorization. Concurrent writes have no snapshot guarantee. Nothing runs when unused. +* Implementation tests must cover tagged round trips (including large integers, time and UUID), unsupported/malformed/tampered/mismatched cursors, Sort/key disagreement, mixed-direction ties, both directions, boundaries and size clamping, plus a package Example of the key function. Execute our pagination and compliant binding/composition tests on in-process SQLite and PostgreSQL CI; do not assert Bob SQL formatting/rendering guarantees. MySQL remains source-feasible until runtime coverage exists.