orm/projects/port-all-tests · pr/orm

GGitHubfeat(sql-orm-client): createAll can skip rows that collide with a unique constraint (#30365)

文件	最后提交记录	最后更新时间
briefs	test(ports): account for first 500 Prisma engine query tests (#29924) ## Linked issue n/a — no ticket for this project batch. ## At a glance ```ts expect(() => query.public.a.select('string').groupBy()).toThrow( expect.objectContaining({ name: 'StructuredError', code: 'ORM.ARGUMENT_INVALID', message: 'Invalid groupBy arguments', }), ); ``` This is one of 201 executable cases derived from the first 500 prisma-engines query tests; the remaining cases receive explicit, reviewed non-port dispositions. ## Summary The engine query inventory was entirely unchecked, so Prisma Next compatibility and unsupported behavior were both implicit. This PR accounts for exactly the first 500 source cases and preserves case 501 as the boundary for subsequent work. ## Decision This PR ships the first engine-query accounting milestone as three explicit outcomes: 1. 180 passing ports across aggregation, scalar data types, distinct queries, relation filters, and field-reference filters. 2. 21 runnable compatibility gaps marked with `it.fails` and documented one-to-one in the canonical failure ledger. 3. 299 reviewed non-port dispositions recorded in 44 source-suite ledgers. All ports use public ORM, SQL-builder, Mongo AST, and contract-authoring surfaces. No production implementation changes are included. ## Reviewer notes - Generated contract artifacts make up most of the diff. The behavior-bearing additions are the 44 `.test.ts` files under [`test/integration/test/ports/engines/queries/`](test/integration/test/ports/engines/queries/). - The source corpus is pinned to `prisma-engines@e922089b7d7502aff4249d5da3420f6fa55fc6ad`. - The cutoff is deliberate: case 500, `queries::filters::ported_filters::str_not_starts_with`, is checked; case 501, `str_ends_with`, and all later cases remain untouched. - Expected failures preserve executable evidence for current differences such as aggregate cursor/pagination handling, JSONB and enum decoding, scalar-list defaults, and cursor semantics. This PR does not fix those production gaps. - Every runnable port was reviewed and executed individually before the final combined validation pass. ## How it fits together 1. Co-located fixtures reproduce each source suite's relevant PostgreSQL or MongoDB schema and retain generated contract artifacts for deterministic execution. 2. Public Prisma Next query surfaces translate the upstream operation while preserving inputs, database-side behavior, and complete assertions. 3. Executable divergences remain in the suite as `it.fails`, paired with precise current-behavior explanations in [`failing.md`](test/integration/test/ports/engines/failing.md). 4. Cases without a faithful public translation are recorded one test per line under [`non-ported/queries`](test/integration/test/ports/engines/non-ported/queries/). 5. [`engines-queries.md`](projects/port-all-tests/checklists/engines-queries.md) links every one of the first 500 source identifiers to exactly one disposition. 6. The new [implementer](projects/port-all-tests/briefs/engine-implementer.md) and [reviewer](projects/port-all-tests/briefs/engine-reviewer.md) briefs codify source-file batch limits, individual execution, fidelity review, and single-writer finalization. ## Behavior changes & evidence - Aggregation cases now run through Prisma Next's public query APIs.* Coverage includes count, average, min/max/sum, group-by, HAVING, and relation counts in [`queries/aggregation`](test/integration/test/ports/engines/queries/aggregation/), with representative evidence in [`group_by.test.ts`](test/integration/test/ports/engines/queries/aggregation/group_by/group_by.test.ts) and [`group_by_having.test.ts`](test/integration/test/ports/engines/queries/aggregation/group_by_having/group_by_having.test.ts). - Scalar and native PostgreSQL behavior is exercised against faithful schemas. The ports cover BigInt, Boolean, Bytes, DateTime, Decimal, enums, Float, Int, JSON, strings, native PostgreSQL types, and relation traversal under [`queries/data_types`](test/integration/test/ports/engines/queries/data_types/), including [`postgres.test.ts`](test/integration/test/ports/engines/queries/data_types/native/postgres/postgres.test.ts). - Filter behavior now includes scalar, relation, list, JSON, and field-reference cases. The public ORM and SQL-builder paths are exercised under [`queries/filters`](test/integration/test/ports/engines/queries/filters/), with field-reference evidence in [`relation_filter.test.ts`](test/integration/test/ports/engines/queries/filters/field_reference/relation_filter/relation_filter.test.ts) and [`json_filter.test.ts`](test/integration/test/ports/engines/queries/filters/field_reference/json_filter/json_filter.test.ts). - Known compatibility differences stay executable and visible. The 21 expected failures are documented in [`failing.md`](test/integration/test/ports/engines/failing.md) and exercised by suites such as [`sum.test.ts`](test/integration/test/ports/engines/queries/aggregation/sum/sum.test.ts) and [`json.test.ts`](test/integration/test/ports/engines/queries/data_types/json/json.test.ts). - Unsupported cases have precise source-level accounting. The 299 entries across [`non-ported/queries`](test/integration/test/ports/engines/non-ported/queries/) distinguish unavailable connectors, APIs, authoring features, and engine-specific protocols instead of treating them as generic skips. ## Compatibility / migration / risk There are no production API, runtime, or migration changes. The review risk is test fidelity: each port must retain the source schema, operation, inputs, and full assertions rather than merely exercise similar behavior. The reviewer protocol and one-to-one checklist accounting are intended to make that fidelity auditable. ## Testing performed - Focused engine suite: 44 files passed; 180 tests passed; 21 expected failures; no type errors. - `pnpm build` — 86 tasks successful. - `cd test/integration && pnpm typecheck` — passed. - `pnpm lint` — 101 tasks successful; informational pre-existing Biome diagnostics only. - `pnpm fixtures:check` — passed with no fixture drift. - `git diff --check` — passed. - Accounting validation — 500 checked, 373 unchecked; 180 PASS, 21 `test.fails`, 299 non-ported; no inboxes remain. ## Skill update n/a — internal test inventory and review workflow only; no user-facing API or CLI surface changes. ## Follow-ups - Continue from case 501 in a separate milestone. - Address the production gaps captured by the 21 expected failures independently from this accounting PR. ## Alternatives considered - Port beyond case 500 in the same PR: not chosen because the fixed boundary keeps the review and accounting milestone auditable. - Mark every non-running case as skipped: not chosen because skips do not distinguish unavailable public surfaces from known executable incompatibilities. - Use internal runtime access or test-only production changes to force ports: not chosen because those translations would not represent behavior available to Prisma Next consumers. - Fix compatibility gaps while porting: not chosen because mixing production fixes into the accounting work would obscure whether each source case was translated faithfully. ## Checklist - [x] All commits are signed off (`git commit -s`) per the [DCO](../CONTRIBUTING.md#developer-certificate-of-origin-dco). - [x] I read [CONTRIBUTING.md](../CONTRIBUTING.md) and the change is scoped to one logical concern. - [x] Tests are updated. - [ ] The PR title is in `TML-NNNN: <sentence-case title>` form — n/a, this work has no Linear ticket. - [x] The Skill update section above is filled in. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - Tests - Expanded PostgreSQL integration coverage for aggregations, grouping, relation counts, distinct queries, filters, field references, relations, and scalar data types. - Added validation for BigInt, Decimal, JSON, enums, bytes, dates, lists, null handling, pagination, and nested relations. - Added regression coverage for relation-filter and one-to-one behaviors. - Improved access to the connected database client in integration tests. - Documentation - Documented unsupported or non-portable scenarios across providers and advanced query patterns. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Steven McClankerton <tatarintsev@prisma.io> Signed-off-by: Steven McClankerton <tatarintsev+clanker@prisma.io> Co-authored-by: Steven McClankerton <tatarintsev@prisma.io>	1 个月前
checklists	feat(sql-orm-client): createAll can skip rows that collide with a unique constraint (#30365) ```ts const rows = [ { id: 1, email: 'ada@example.com' }, { id: 2, email: 'ada@example.com' }, // same email as row 1 { id: 3, email: 'grace@example.com' }, ]; const inserted = await db.User.createAll(rows, { onConflict: 'skip' }); // [{ id: 1, ... }, { id: 3, ... }] row 2 was skipped by the database const count = await db.User.createAndCount(rows, { onConflict: 'skip' }); // 2 ``` Before this PR, that call threw a unique violation and inserted nothing. Now the database skips the colliding row and the ORM returns only what was actually written. ## The decision `createAll` and `createAndCount` accept an options object. Its only option is `onConflict: 'skip'`, optionally with `conflictOn: ['email']` to name which unique constraint counts as "a conflict". The SQL is `INSERT ... ON CONFLICT DO NOTHING`, or `ON CONFLICT ("email") DO NOTHING` when a target is named. Prisma 7 called this `createMany({ skipDuplicates: true })`. We promised a user in `#prisma-8` on 2026-09-09 that Prisma 8 would get an equivalent. The scorecard listed it as deliberately absent, which contradicted that promise. ## How it works, from the SQL up The SQL layer already knew how to do this, almost. `upsert` uses `ON CONFLICT` today, so the SQL AST has an `InsertOnConflict` node and both renderers emit the clause. The one restriction was that a conflict clause had to name at least one column. Postgres and SQLite both accept `ON CONFLICT DO NOTHING` with no target, meaning "skip on any unique constraint", and that is what `skipDuplicates` did. So the AST gains a targetless form that is valid only with `DO NOTHING`. A targetless `DO UPDATE SET` makes no sense and both renderers still reject it. What comes back. `createAll` uses `RETURNING`, so it naturally yields only the rows the database inserted. `createAndCount` was a different story: it returned the length of the input array, not anything the database said. That was already a bug (a trigger could drop a row and the count would lie), and it would have made the count wrong for every skipped row. It now reads the driver's affected-row count, the way `updateAndCount` and `deleteAndCount` always have. That fix is the second commit and stands on its own. Why SQLite needs a little more. SQLite cannot take `DEFAULT` as a value in a multi-row insert, so when rows have different column sets the ORM already splits one `createAll` into one statement per column signature. The conflict clause goes on every one of those statements, and `createAndCount` sums the counts. Why this is behind capabilities. Prisma 8 never asks "which database is this". Instead, adapters report what they can do, the emitted `contract.json` records those capabilities, and features check for a capability key at the point of use. Two new keys: - `sql.insertOnConflictSkip`: the adapter can skip rows on a unique conflict at all. - `sql.insertOnConflictWithoutTarget`: it can do so without naming the constraint. Postgres and SQLite report both. The split exists for targets we plan to add. MySQL has no `ON CONFLICT` but can skip with `INSERT IGNORE`. SQL Server has neither and can only express "skip" as a `MERGE` or `WHERE NOT EXISTS` on an explicit key, so it will report the first key and not the second. Without the first key the option is refused with `ORM.CAPABILITY_MISSING`; without the second, only the untargeted form is refused. That is also why `conflictOn` is part of the API from day one rather than something bolted on when SQL Server arrives. What is refused on purpose. Multi-table inheritance variants: a variant row is a base-table insert followed by a variant-table insert per row, and a skipped variant insert would leave an orphan base row. Refused with `ORM.OPERATION_UNSUPPORTED` until we do savepoint-per-row. `create()` does not take the option; a single row that may or may not be inserted is `upsert` with an empty update, which already exists. Mongo is untouched. ## What is in the diff - `packages/2-sql/4-lanes/relational-core`: `InsertOnConflict.doNothing()` for the targetless form. - `packages/3-targets/6-adapters/{postgres,sqlite}`: render the targetless clause; report the two capability keys in the runtime profile and in the map that the emitter copies into `contract.json`. - `packages/3-extensions/sql-orm-client`: the option on `createAll` and `createAndCount`, the capability and MTI checks, `conflictOn` field-to-column mapping through the same path `upsert` uses, and the `createAndCount` count fix. - `docs/reference/capabilities.md`, the sql-orm-client README, JSDoc on both methods, the scorecard row, and an upgrade note. - `projects/createall-conflict-skip/`: the spec, design notes, and plan this was built from. Transient; deleted at close-out per the projects convention. - 473 regenerated `contract.json` / `contract.d.ts` fixtures. See the next section before you scroll through them. ## About the regenerated contracts Every emitted contract carries the adapter's capability map, so adding two keys touches every fixture in the repo. I checked all 473 files mechanically: the only change in any of them is the two added keys, and no hash moved. Eighteen of them are migration snapshots under `migrations/snapshots/`. Our convention says snapshots are written once, but the regen scripts that `pnpm fixtures:check` runs rewrite snapshot contracts unconditionally, so leaving them out fails the check. They are included; a contract without the keys still loads fine and simply refuses the option. Making the regen scripts leave existing snapshots alone is a separate follow-up. Users on an existing contract must re-emit it before they can use the option. The upgrade note under `upgrade-instructions/pending/` says so. ## How it is tested - Postgres integration, `test/integration/test/sql-orm-client/create-on-conflict.test.ts`: collision against an existing row, targeted and untargeted, every row colliding, a collision on a constraint other than the named one still raising, both capability refusals, and a batch whose rows collide with each other. Each goes red if the clause is dropped. - SQLite end to end: the split path with a collision in one of two statements, and the within-batch case. - Two Prisma engine cases (`create_many_no_error_skip_dup` and its `createManyAndReturn` variant) ported faithfully under `test/integration/test/ports/engines/writes/top_level_mutations/create_many/` and ticked in the port checklist. - Unit tier: the AST node, both renderers with exact SQL strings, plan compilation, collection behaviour including the old callback-in-second-position call form, and every structured error code. - The `createAndCount` fix: a `BEFORE INSERT` trigger drops one of three rows on Postgres and the count comes back as 2. ## Alternatives considered - `skipDuplicates: true`, the Prisma 7 name. Familiar, but a boolean cannot carry a conflict target, and SQL Server needs one. Rejected. - `onConflict: 'doNothing'`. Serhii's suggestion; matches the SQL. Kept `'skip'` because it is target-neutral: MySQL spells this `INSERT IGNORE` and SQL Server `MERGE`, and `'skip'` names what the caller observes. Neither of us felt strongly. - Letting `onConflict` grow an `update` value later. No. `upsert` owns update-on-conflict; a batch form would be its own method. - A separate method such as `createAllIgnoringConflicts`. Doubles with `createAndCount`. Rejected. - A chained modifier, `db.User.onConflict('skip').createAll(rows)`. Puts a write-only flag into builder state that `find`, `select`, and `include` share. Rejected. - Always requiring `conflictOn`. Most portable, but forces every Postgres and SQLite user to name a constraint for the common case. The second capability key makes the requirement target-specific instead. - Supporting MTI variants with a savepoint per row. Correct but a second transaction concern inside an experimental surface. Deferred. Supersedes #30363 and #30364, which were these commits split for stacked review. No Linear issue exists for this work yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	5 天前
inventory.md	Port prisma & prisma-engines test corpus into prisma-next (488 accounted) (#1035) ## Linked issue n/a — no Linear ticket for this project yet. The project spec, scope contract, and accounting ledgers live in-repo under [`projects/port-all-tests/`](projects/port-all-tests/spec.md). ## At a glance A faithful port reproduces the upstream test's schema, query, and assertions through prisma-next's public API — and every source test lands in exactly one of three buckets: passing, `test.fails` (faithful but hits a real gap), or non-ported (the API genuinely cannot express it). ```ts // passing — upstream functional/handle-int-overflow: an out-of-range Int is rejected it('integer overflow', () => withPostgresPort<Contract>({ contractJson }, async ({ db }) => { await expect(db.public.Entry.create({ int: 1e20 })).rejects.toThrow(); })); // test.fails — faithful call, genuine prisma-next gap (one failing.md line each) it.fails('include _count should fail', () => withUMaR(async ({ db }) => { // @ts-expect-error `_count` is not a relation await expect(db.public.User.include('_count').where({}).updateAll({ name: 'y' })).rejects.toThrow(); })); ``` Non-portable tests never get a bent-green stand-in — they get one accounted line in `non-ported.md` naming the exact source location and the precise reason (e.g. "no fluent relation-traversal API", "`Prisma.JsonNull` sentinel has no equivalent", "SQLite-only suite"). ## Decision This PR delivers a behavioral-compatibility corpus derived from the two upstream Prisma test suites, ported into the existing integration-test package. It ships three things: 1. The ported corpus — vitest tests under `test/integration/test/ports/{prisma,engines}/` that re-express upstream Prisma tests against prisma-next's public API (the `orm()`/`Collection` surface, `sql()`, the CLI/Control API), with PSL fixtures pushed through prisma-next's own plan→apply (no hand-written DDL). 2. The accounting ledger — a checklist corpus (one checkbox per source test) plus `non-ported.md` / `failing.md`, so "does prisma-next cover Prisma's behavior?" becomes a measurable, test-granular map instead of a guess. 3. The execution methodology — an enumerate → implement → review-gate loop with mandatory verbatim implementer/reviewer briefs, captured under `projects/port-all-tests/` so the remaining tranches port the same way. No production code changes. Gaps the corpus surfaces are recorded (`test.fails` + `failing.md`, or `non-ported.md`) — never worked around in a test and never patched in framework code. That is the whole point: the ledger stays honest. ## How it fits together 1. Enumerate first. Every in-scope upstream test is enumerated up-front into [`projects/port-all-tests/checklists/`](projects/port-all-tests/checklists/README.md) — one checkbox line per source test with its connector/provider tags. The checklists are both the work-list and the accounting ledger; a suite not listed there is not silently in play. 2. Port faithfully, or don't port. The faithfulness contract ([`spec.md` § "No workarounds"](projects/port-all-tests/spec.md)) is the hard gate: same schema, logically the same query, the same assertions — runtime and type-level (`expectTypeOf`/`@ts-expect-error` ported inline). API-shape translation (`findMany({where})` → `.where().all()`) is allowed; swapping the mechanism/input/asserted-result to force a green is forbidden and becomes `non-ported`/`test.fails` instead. 3. Fleet + gate. Batches (≤10 source files) are ported by Sonnet implementer sub-agents dispatched with [`briefs/implementer.md`](projects/port-all-tests/briefs/implementer.md) verbatim, then independently re-run (`pnpm test`/`typecheck`/`lint`) and gated by an Opus reviewer dispatched with [`briefs/reviewer.md`](projects/port-all-tests/briefs/reviewer.md) that diffs each port against source and challenges every disposition. Only the reviewer checks a box. 4. Account at every merge. Each batch lands its `non-ported.md`/`failing.md`/checklist updates with its tests; `failing.md` entries match the `test.fails` markers one-to-one. Fixtures author contracts exclusively through public PSL and are pushed via the same mechanism `db init` uses. ## What lands in this PR The branch is the project to date — 11 commits, foundation through the latest tranche: \| Commit \| What it adds \| \| --- \| --- \| \| `1b84d34a5` \| Foundation + first batch (distinct, default-selection, create-default-date, blog-update) \| \| `0c9c95d72` \| Rework foundation to PSL + contract-push (no hand-written DDL) \| \| `3602f50a5` \| Mongo harness (`mongodb-memory-server`) + corrected SQL/mongo batch \| \| `9c0057059` \| Batch C+D — methods, filters, field-reference, extended-where \| \| `fe672eb3d` \| Batch E+F — defaults, aggregations, decimals, json, relation filters \| \| `16498c292` \| Wave G–K + hardened the No-workarounds spec gate \| \| `ff4a22988` \| Audit-driven faithfulness pass (restore inline type assertions, fix schema-simplifications) \| \| `15d8ab0ed`, `9a241331c` \| Mandatory implementer/reviewer briefs; DB-schema-must-match-original as an explicit reviewer gate \| \| `dcf996704` \| Close audit debt — accounting parity, tightened assertions, spec/brief reconciliation \| \| `14de46273` \| Wave L + Wave 2 — scalars/json, writes-returning, referential SetDefault, regressions (+244 accounted) \| ## Reviewer notes - Large by design, and test-only. This is a corpus PR; per the repo's optimize-for-reviewer-time guidance it's kept as one reviewable project unit rather than dozens of micro-PRs. There are zero changes under `packages/` — read it as tests + fixtures + ledgers. - The non-ported count is high on purpose, and it's the part to actually check.** A contract-first ORM has no analogue for many Prisma-client-only features (fluent relation traversal, `$extends`, `Prisma.skip`/`omit`/`validator`/`views`, `Prisma.JsonNull`/`DbNull` sentinels, `ScalarFieldRef`, fulltext `search`/`_relevance`, `NULLS FIRST/LAST`, JSON-path filters), and unsupported-provider-only suites (mysql/sqlserver/cockroach/sqlite/mongo-only) are recorded per-test. The risk here isn't a red test — it's a wrong non-port reason, so spot-check `non-ported.md` lines against the API, not just the green suites. - The review gate already caught real wrong-dispositions — a filter-count M:N suite that a fixture-shape artifact made look inexpressible (it's a passing nested port + 3 genuine `test.fails`), an int-overflow suite that does reject at runtime, and createManyAndReturn `_count` cases that are inline `@ts-expect-error` ports, not non-ports. Those reworked areas are the highest-value spot-checks. - The 30 `test.fails` are genuine prisma-next gaps, each with a `failing.md` line (N:M `include(...).count()` ignoring the `through` descriptor; cross-namespace `some()` emitting an unqualified table name; inclusive-vs-exclusive cursor; enum-array emitter gap; `include('_count')` type-rejected but ignored at runtime; bytes-upsert conflict reload). Each flips to green if the gap is fixed — that's the litmus separating a faithful `test.fails` from a workaround. - Project artifacts are left on disk deliberately under `projects/port-all-tests/` (spec, briefs, checklists, an audit trail) so this methodology is reviewable. Close-out migrates the long-lived pieces into `docs/` and deletes the transient project dir at project DoD; this PR is not the close-out. - Sources are pinned, not vendored — `prisma/prisma@a6d0155` and `prisma/prisma-engines@e922089` (checked out at `/tmp/prisma`, `/tmp/prisma-engines`); the corpus imports only public prisma-next package exports. ## Testing performed Run on the final HEAD: - `pnpm test test/ports/` (with `MONGOMS_DISTRO=ubuntu-22.04` for the mongo suites) — 43 files, 177 passed + 30 expected-fail, 0 type errors. - `pnpm typecheck` (`tsc --noEmit` over the package — the real type gate; vitest's esbuild hides type errors) — exit 0. - `pnpm lint` — exit 0 (the lone biome "info" is a pre-existing schema-version notice, unrelated). - Accounting cross-check: 488 functional checklist boxes checked; `non-ported.md` 515 entries; `failing.md` 30 entries matching the 30 `test.fails` markers one-to-one. ## Skill update n/a — test-only project. No user-facing surface (CLI, public TypeScript API, config, error codes) changes; the spec/briefs under `projects/port-all-tests/` are project methodology, not an installable skill. ## Follow-ups - The remaining checklist tranches (engines query/write/migration/introspection, prisma migrate/CLI, legacy suites, more functional + issues) continue on this branch under the same gate. - Each `test.fails` / `non-ported` cluster is a precise, test-granular feature-gap map for future, separate framework projects (N:M include-count `through`; cross-namespace subquery qualification; atomic update operators; `_count`-in-`include`; JSON-path filters; SQLite integration harness). - Project close-out (migrate long-lived docs into `docs/`, delete `projects/port-all-tests/`) is deferred to project DoD. ## Alternatives considered - Re-create the upstream harnesses (provider-matrix generator, schema-templating engine, insta-snapshot runner). Rejected: each port is a plain vitest test on the repo's existing PGlite/mongo-memory-server harnesses; snapshots become explicit `toEqual`. Less machinery, and the port reads like the rest of the suite. - Emulate unsupported databases on Postgres (mysql/mssql/cockroach/vitess/D1-only tests). Rejected: that would fabricate coverage. Provider-exclusive tests get individual `non-ported.md` lines with the provider tag as evidence. - Sibling `.test-d.ts` files for type assertions. Rejected in favor of inline `expectTypeOf`/`@ts-expect-error` in the same `it()` — it keeps a single upstream test as a single test and preserves the runtime+type coupling; `pnpm typecheck` is the gate. - Fix the gaps the ports reveal in framework code. Out of scope by construction — this project measures behavior; fixing it is future, separate work. Recording a gap as `test.fails`/`non-ported` is the deliverable, not a failure. ## Checklist - [x] All commits are signed off (`git commit -s`) per the DCO. - [x] I read CONTRIBUTING.md and the change is scoped to one logical concern (the test-porting corpus). - [x] Tests are updated — this PR is tests; ledgers land with them. - [ ] The PR title is in `TML-NNNN: <title>` form — n/a, no Linear ticket exists for this project yet. - [x] The Skill update section is filled in (`n/a — internal only`). --------- Signed-off-by: Serhii Tatarintsev <tatarintsev@prisma.io> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2 个月前
spec.md	test(ports): account for first 500 Prisma engine query tests (#29924) ## Linked issue n/a — no ticket for this project batch. ## At a glance ```ts expect(() => query.public.a.select('string').groupBy()).toThrow( expect.objectContaining({ name: 'StructuredError', code: 'ORM.ARGUMENT_INVALID', message: 'Invalid groupBy arguments', }), ); ``` This is one of 201 executable cases derived from the first 500 prisma-engines query tests; the remaining cases receive explicit, reviewed non-port dispositions. ## Summary The engine query inventory was entirely unchecked, so Prisma Next compatibility and unsupported behavior were both implicit. This PR accounts for exactly the first 500 source cases and preserves case 501 as the boundary for subsequent work. ## Decision This PR ships the first engine-query accounting milestone as three explicit outcomes: 1. 180 passing ports across aggregation, scalar data types, distinct queries, relation filters, and field-reference filters. 2. 21 runnable compatibility gaps marked with `it.fails` and documented one-to-one in the canonical failure ledger. 3. 299 reviewed non-port dispositions recorded in 44 source-suite ledgers. All ports use public ORM, SQL-builder, Mongo AST, and contract-authoring surfaces. No production implementation changes are included. ## Reviewer notes - Generated contract artifacts make up most of the diff. The behavior-bearing additions are the 44 `.test.ts` files under [`test/integration/test/ports/engines/queries/`](test/integration/test/ports/engines/queries/). - The source corpus is pinned to `prisma-engines@e922089b7d7502aff4249d5da3420f6fa55fc6ad`. - The cutoff is deliberate: case 500, `queries::filters::ported_filters::str_not_starts_with`, is checked; case 501, `str_ends_with`, and all later cases remain untouched. - Expected failures preserve executable evidence for current differences such as aggregate cursor/pagination handling, JSONB and enum decoding, scalar-list defaults, and cursor semantics. This PR does not fix those production gaps. - Every runnable port was reviewed and executed individually before the final combined validation pass. ## How it fits together 1. Co-located fixtures reproduce each source suite's relevant PostgreSQL or MongoDB schema and retain generated contract artifacts for deterministic execution. 2. Public Prisma Next query surfaces translate the upstream operation while preserving inputs, database-side behavior, and complete assertions. 3. Executable divergences remain in the suite as `it.fails`, paired with precise current-behavior explanations in [`failing.md`](test/integration/test/ports/engines/failing.md). 4. Cases without a faithful public translation are recorded one test per line under [`non-ported/queries`](test/integration/test/ports/engines/non-ported/queries/). 5. [`engines-queries.md`](projects/port-all-tests/checklists/engines-queries.md) links every one of the first 500 source identifiers to exactly one disposition. 6. The new [implementer](projects/port-all-tests/briefs/engine-implementer.md) and [reviewer](projects/port-all-tests/briefs/engine-reviewer.md) briefs codify source-file batch limits, individual execution, fidelity review, and single-writer finalization. ## Behavior changes & evidence - Aggregation cases now run through Prisma Next's public query APIs.* Coverage includes count, average, min/max/sum, group-by, HAVING, and relation counts in [`queries/aggregation`](test/integration/test/ports/engines/queries/aggregation/), with representative evidence in [`group_by.test.ts`](test/integration/test/ports/engines/queries/aggregation/group_by/group_by.test.ts) and [`group_by_having.test.ts`](test/integration/test/ports/engines/queries/aggregation/group_by_having/group_by_having.test.ts). - Scalar and native PostgreSQL behavior is exercised against faithful schemas. The ports cover BigInt, Boolean, Bytes, DateTime, Decimal, enums, Float, Int, JSON, strings, native PostgreSQL types, and relation traversal under [`queries/data_types`](test/integration/test/ports/engines/queries/data_types/), including [`postgres.test.ts`](test/integration/test/ports/engines/queries/data_types/native/postgres/postgres.test.ts). - Filter behavior now includes scalar, relation, list, JSON, and field-reference cases. The public ORM and SQL-builder paths are exercised under [`queries/filters`](test/integration/test/ports/engines/queries/filters/), with field-reference evidence in [`relation_filter.test.ts`](test/integration/test/ports/engines/queries/filters/field_reference/relation_filter/relation_filter.test.ts) and [`json_filter.test.ts`](test/integration/test/ports/engines/queries/filters/field_reference/json_filter/json_filter.test.ts). - Known compatibility differences stay executable and visible. The 21 expected failures are documented in [`failing.md`](test/integration/test/ports/engines/failing.md) and exercised by suites such as [`sum.test.ts`](test/integration/test/ports/engines/queries/aggregation/sum/sum.test.ts) and [`json.test.ts`](test/integration/test/ports/engines/queries/data_types/json/json.test.ts). - Unsupported cases have precise source-level accounting. The 299 entries across [`non-ported/queries`](test/integration/test/ports/engines/non-ported/queries/) distinguish unavailable connectors, APIs, authoring features, and engine-specific protocols instead of treating them as generic skips. ## Compatibility / migration / risk There are no production API, runtime, or migration changes. The review risk is test fidelity: each port must retain the source schema, operation, inputs, and full assertions rather than merely exercise similar behavior. The reviewer protocol and one-to-one checklist accounting are intended to make that fidelity auditable. ## Testing performed - Focused engine suite: 44 files passed; 180 tests passed; 21 expected failures; no type errors. - `pnpm build` — 86 tasks successful. - `cd test/integration && pnpm typecheck` — passed. - `pnpm lint` — 101 tasks successful; informational pre-existing Biome diagnostics only. - `pnpm fixtures:check` — passed with no fixture drift. - `git diff --check` — passed. - Accounting validation — 500 checked, 373 unchecked; 180 PASS, 21 `test.fails`, 299 non-ported; no inboxes remain. ## Skill update n/a — internal test inventory and review workflow only; no user-facing API or CLI surface changes. ## Follow-ups - Continue from case 501 in a separate milestone. - Address the production gaps captured by the 21 expected failures independently from this accounting PR. ## Alternatives considered - Port beyond case 500 in the same PR: not chosen because the fixed boundary keeps the review and accounting milestone auditable. - Mark every non-running case as skipped: not chosen because skips do not distinguish unavailable public surfaces from known executable incompatibilities. - Use internal runtime access or test-only production changes to force ports: not chosen because those translations would not represent behavior available to Prisma Next consumers. - Fix compatibility gaps while porting: not chosen because mixing production fixes into the accounting work would obscure whether each source case was translated faithfully. ## Checklist - [x] All commits are signed off (`git commit -s`) per the [DCO](../CONTRIBUTING.md#developer-certificate-of-origin-dco). - [x] I read [CONTRIBUTING.md](../CONTRIBUTING.md) and the change is scoped to one logical concern. - [x] Tests are updated. - [ ] The PR title is in `TML-NNNN: <sentence-case title>` form — n/a, this work has no Linear ticket. - [x] The Skill update section above is filled in. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - Tests - Expanded PostgreSQL integration coverage for aggregations, grouping, relation counts, distinct queries, filters, field references, relations, and scalar data types. - Added validation for BigInt, Decimal, JSON, enums, bytes, dates, lists, null handling, pagination, and nested relations. - Added regression coverage for relation-filter and one-to-one behaviors. - Improved access to the connected database client in integration tests. - Documentation - Documented unsupported or non-portable scenarios across providers and advanced query patterns. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Steven McClankerton <tatarintsev@prisma.io> Signed-off-by: Steven McClankerton <tatarintsev+clanker@prisma.io> Co-authored-by: Steven McClankerton <tatarintsev@prisma.io>	1 个月前
trace.jsonl	Port prisma & prisma-engines test corpus into prisma-next (488 accounted) (#1035) ## Linked issue n/a — no Linear ticket for this project yet. The project spec, scope contract, and accounting ledgers live in-repo under [`projects/port-all-tests/`](projects/port-all-tests/spec.md). ## At a glance A faithful port reproduces the upstream test's schema, query, and assertions through prisma-next's public API — and every source test lands in exactly one of three buckets: passing, `test.fails` (faithful but hits a real gap), or non-ported (the API genuinely cannot express it). ```ts // passing — upstream functional/handle-int-overflow: an out-of-range Int is rejected it('integer overflow', () => withPostgresPort<Contract>({ contractJson }, async ({ db }) => { await expect(db.public.Entry.create({ int: 1e20 })).rejects.toThrow(); })); // test.fails — faithful call, genuine prisma-next gap (one failing.md line each) it.fails('include _count should fail', () => withUMaR(async ({ db }) => { // @ts-expect-error `_count` is not a relation await expect(db.public.User.include('_count').where({}).updateAll({ name: 'y' })).rejects.toThrow(); })); ``` Non-portable tests never get a bent-green stand-in — they get one accounted line in `non-ported.md` naming the exact source location and the precise reason (e.g. "no fluent relation-traversal API", "`Prisma.JsonNull` sentinel has no equivalent", "SQLite-only suite"). ## Decision This PR delivers a behavioral-compatibility corpus derived from the two upstream Prisma test suites, ported into the existing integration-test package. It ships three things: 1. The ported corpus — vitest tests under `test/integration/test/ports/{prisma,engines}/` that re-express upstream Prisma tests against prisma-next's public API (the `orm()`/`Collection` surface, `sql()`, the CLI/Control API), with PSL fixtures pushed through prisma-next's own plan→apply (no hand-written DDL). 2. The accounting ledger — a checklist corpus (one checkbox per source test) plus `non-ported.md` / `failing.md`, so "does prisma-next cover Prisma's behavior?" becomes a measurable, test-granular map instead of a guess. 3. The execution methodology — an enumerate → implement → review-gate loop with mandatory verbatim implementer/reviewer briefs, captured under `projects/port-all-tests/` so the remaining tranches port the same way. No production code changes. Gaps the corpus surfaces are recorded (`test.fails` + `failing.md`, or `non-ported.md`) — never worked around in a test and never patched in framework code. That is the whole point: the ledger stays honest. ## How it fits together 1. Enumerate first. Every in-scope upstream test is enumerated up-front into [`projects/port-all-tests/checklists/`](projects/port-all-tests/checklists/README.md) — one checkbox line per source test with its connector/provider tags. The checklists are both the work-list and the accounting ledger; a suite not listed there is not silently in play. 2. Port faithfully, or don't port. The faithfulness contract ([`spec.md` § "No workarounds"](projects/port-all-tests/spec.md)) is the hard gate: same schema, logically the same query, the same assertions — runtime and type-level (`expectTypeOf`/`@ts-expect-error` ported inline). API-shape translation (`findMany({where})` → `.where().all()`) is allowed; swapping the mechanism/input/asserted-result to force a green is forbidden and becomes `non-ported`/`test.fails` instead. 3. Fleet + gate. Batches (≤10 source files) are ported by Sonnet implementer sub-agents dispatched with [`briefs/implementer.md`](projects/port-all-tests/briefs/implementer.md) verbatim, then independently re-run (`pnpm test`/`typecheck`/`lint`) and gated by an Opus reviewer dispatched with [`briefs/reviewer.md`](projects/port-all-tests/briefs/reviewer.md) that diffs each port against source and challenges every disposition. Only the reviewer checks a box. 4. Account at every merge. Each batch lands its `non-ported.md`/`failing.md`/checklist updates with its tests; `failing.md` entries match the `test.fails` markers one-to-one. Fixtures author contracts exclusively through public PSL and are pushed via the same mechanism `db init` uses. ## What lands in this PR The branch is the project to date — 11 commits, foundation through the latest tranche: \| Commit \| What it adds \| \| --- \| --- \| \| `1b84d34a5` \| Foundation + first batch (distinct, default-selection, create-default-date, blog-update) \| \| `0c9c95d72` \| Rework foundation to PSL + contract-push (no hand-written DDL) \| \| `3602f50a5` \| Mongo harness (`mongodb-memory-server`) + corrected SQL/mongo batch \| \| `9c0057059` \| Batch C+D — methods, filters, field-reference, extended-where \| \| `fe672eb3d` \| Batch E+F — defaults, aggregations, decimals, json, relation filters \| \| `16498c292` \| Wave G–K + hardened the No-workarounds spec gate \| \| `ff4a22988` \| Audit-driven faithfulness pass (restore inline type assertions, fix schema-simplifications) \| \| `15d8ab0ed`, `9a241331c` \| Mandatory implementer/reviewer briefs; DB-schema-must-match-original as an explicit reviewer gate \| \| `dcf996704` \| Close audit debt — accounting parity, tightened assertions, spec/brief reconciliation \| \| `14de46273` \| Wave L + Wave 2 — scalars/json, writes-returning, referential SetDefault, regressions (+244 accounted) \| ## Reviewer notes - Large by design, and test-only. This is a corpus PR; per the repo's optimize-for-reviewer-time guidance it's kept as one reviewable project unit rather than dozens of micro-PRs. There are zero changes under `packages/` — read it as tests + fixtures + ledgers. - The non-ported count is high on purpose, and it's the part to actually check.** A contract-first ORM has no analogue for many Prisma-client-only features (fluent relation traversal, `$extends`, `Prisma.skip`/`omit`/`validator`/`views`, `Prisma.JsonNull`/`DbNull` sentinels, `ScalarFieldRef`, fulltext `search`/`_relevance`, `NULLS FIRST/LAST`, JSON-path filters), and unsupported-provider-only suites (mysql/sqlserver/cockroach/sqlite/mongo-only) are recorded per-test. The risk here isn't a red test — it's a wrong non-port reason, so spot-check `non-ported.md` lines against the API, not just the green suites. - The review gate already caught real wrong-dispositions — a filter-count M:N suite that a fixture-shape artifact made look inexpressible (it's a passing nested port + 3 genuine `test.fails`), an int-overflow suite that does reject at runtime, and createManyAndReturn `_count` cases that are inline `@ts-expect-error` ports, not non-ports. Those reworked areas are the highest-value spot-checks. - The 30 `test.fails` are genuine prisma-next gaps, each with a `failing.md` line (N:M `include(...).count()` ignoring the `through` descriptor; cross-namespace `some()` emitting an unqualified table name; inclusive-vs-exclusive cursor; enum-array emitter gap; `include('_count')` type-rejected but ignored at runtime; bytes-upsert conflict reload). Each flips to green if the gap is fixed — that's the litmus separating a faithful `test.fails` from a workaround. - Project artifacts are left on disk deliberately under `projects/port-all-tests/` (spec, briefs, checklists, an audit trail) so this methodology is reviewable. Close-out migrates the long-lived pieces into `docs/` and deletes the transient project dir at project DoD; this PR is not the close-out. - Sources are pinned, not vendored — `prisma/prisma@a6d0155` and `prisma/prisma-engines@e922089` (checked out at `/tmp/prisma`, `/tmp/prisma-engines`); the corpus imports only public prisma-next package exports. ## Testing performed Run on the final HEAD: - `pnpm test test/ports/` (with `MONGOMS_DISTRO=ubuntu-22.04` for the mongo suites) — 43 files, 177 passed + 30 expected-fail, 0 type errors. - `pnpm typecheck` (`tsc --noEmit` over the package — the real type gate; vitest's esbuild hides type errors) — exit 0. - `pnpm lint` — exit 0 (the lone biome "info" is a pre-existing schema-version notice, unrelated). - Accounting cross-check: 488 functional checklist boxes checked; `non-ported.md` 515 entries; `failing.md` 30 entries matching the 30 `test.fails` markers one-to-one. ## Skill update n/a — test-only project. No user-facing surface (CLI, public TypeScript API, config, error codes) changes; the spec/briefs under `projects/port-all-tests/` are project methodology, not an installable skill. ## Follow-ups - The remaining checklist tranches (engines query/write/migration/introspection, prisma migrate/CLI, legacy suites, more functional + issues) continue on this branch under the same gate. - Each `test.fails` / `non-ported` cluster is a precise, test-granular feature-gap map for future, separate framework projects (N:M include-count `through`; cross-namespace subquery qualification; atomic update operators; `_count`-in-`include`; JSON-path filters; SQLite integration harness). - Project close-out (migrate long-lived docs into `docs/`, delete `projects/port-all-tests/`) is deferred to project DoD. ## Alternatives considered - Re-create the upstream harnesses (provider-matrix generator, schema-templating engine, insta-snapshot runner). Rejected: each port is a plain vitest test on the repo's existing PGlite/mongo-memory-server harnesses; snapshots become explicit `toEqual`. Less machinery, and the port reads like the rest of the suite. - Emulate unsupported databases on Postgres (mysql/mssql/cockroach/vitess/D1-only tests). Rejected: that would fabricate coverage. Provider-exclusive tests get individual `non-ported.md` lines with the provider tag as evidence. - Sibling `.test-d.ts` files for type assertions. Rejected in favor of inline `expectTypeOf`/`@ts-expect-error` in the same `it()` — it keeps a single upstream test as a single test and preserves the runtime+type coupling; `pnpm typecheck` is the gate. - Fix the gaps the ports reveal in framework code. Out of scope by construction — this project measures behavior; fixing it is future, separate work. Recording a gap as `test.fails`/`non-ported` is the deliverable, not a failure. ## Checklist - [x] All commits are signed off (`git commit -s`) per the DCO. - [x] I read CONTRIBUTING.md and the change is scoped to one logical concern (the test-porting corpus). - [x] Tests are updated — this PR is tests; ledgers land with them. - [ ] The PR title is in `TML-NNNN: <title>` form — n/a, no Linear ticket exists for this project yet. - [x] The Skill update section is filled in (`n/a — internal only`). --------- Signed-off-by: Serhii Tatarintsev <tatarintsev@prisma.io> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2 个月前