| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat: move webhook_url override to top-level field. (#6778) > This is a breaking change with low surface impact After a lot of discussion we believe that `connection_config` was not the right place for `webhook_url`. We are trying to move quick while it's still not massively adopted (currently 2 accounts, using it incorrectly as the main `webhook_url` setting). This moves the setting to a top-level property on the connection called `webhook_url_override`. I also considered moving it to `overrides` on the connect_session, but the `overrides` object is keyed by integration, which doesn't really make sense for `webhook_url`. So I'm proposing this top-level alternative instead. Please look at the docs updates to understand the user-level change. --------- Co-authored-by: Cursor <cursoragent@cursor.com> | 1 个月前 | |
feat(functions): upsert schedules if needed when deploying functions (#7525) Since there is no atomic transaction between function entries in main db and schedules in orchestrator, we commit the main db function entries before upserting the schedules in orchestrator. The consequence is an orchestrator failure might lead to missing schedules. The opposite workflow (upserting schedules before committing the main db) was discarded because it would lead to orphan schedules that exists even though a function deployment failed, which is a worst outcome. The missing schedules problem is minimized by making deployment self-heal schedules. To achieve this the POST /recurring orchestrator endpoints doesn't fail anymore when schedule already exists and behave like an upsert. The PR also refactors some orchestrator endpoints and internal functions to accept array instead of single item in order to minimize the number of requests. The change is backward compatible but requires orchestrator to be deployed first. Note: scheduled functions execution is still not implemented. That's next PR <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/NangoHQ/nango/pull/7525?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> | 2 天前 | |
feat(tags): Add backfill script/migration for end user tags -> connection tags (#5342) <!-- Describe the problem and your solution --> This query will update `114 282` rows in production. I will run it asynchronously using the migrate script in our cloud, and let knex automatically migrate connections for on-prem/self-hosted deployments. There are a few assumptions in this query to make it simpler, but I did run the queries in prod DB to verify we don't have to care about them - It doesn't truncate keys longer than 64 characters or values longer than 250 characters (which would be invalid new tags) because there are no existing keys or values that long - It doesn't validate that it won't create more than 10 tags because there are no end users with that many (max is 4) - It doesn't validate invalid key format because all existing end user tags are valid <!-- Issue ticket number and link (if applicable) --> <!-- Testing instructions (skip if just adding/editing providers) --> <!-- Summary by @propel-code-bot --> --- **Backfill connection tags and harden tag sanitization** Introduces a reusable SQL CTE pipeline and operational tooling to backfill `_nango_connections.tags` from existing end-user metadata while aligning runtime tag generation with the same constraints. Adds a non-transactional Knex migration, an operational `migrate.ts` runner with optional migration marking, a local seeding helper, and an end-to-end integration test that exercises truncation, invalid keys, tag precedence, and connections lacking end users. Updates `buildTagsFromEndUser` to normalize keys/values, enforce length/count limits, emit structured telemetry about dropped or truncated tags, and fall back to base tags when custom data is invalid so online ingestion matches the migration behavior. <details> <summary><strong>Key Changes</strong></summary> • Added SQL helper `packages/database/lib/migration-helpers/backfillConnectionTagsSql.cjs` that normalizes base tags, sanitizes end-user tag entries, enforces 10-key limits, merges with existing connection tags, and returns summary metrics for operational visibility. • Registered non-transactional migration `packages/database/lib/migrations/20260128120000_backfill_connection_tags.cjs` plus `scripts/one-off/connection-tags-migration/migrate.ts` (with `--mark-migration`) and companion README/seed tooling for cloud and self-hosted deployments. • Created integration test `packages/database/lib/backfillConnectionTags.integration.test.ts` with inline seed helpers covering special characters, precedence, truncation, invalid formats, over-limit keys, and connections lacking end users. • Expanded `packages/shared/lib/services/endUser.service.ts` to truncate/validate tags, log structured telemetry, drop custom tags when exceeding limits, and fall back gracefully if custom tags fail schema validation, updating unit tests accordingly. </details> <details> <summary><strong>Affected Areas</strong></summary> • `packages/database/lib/migration-helpers/backfillConnectionTagsSql.cjs` • `packages/database/lib/migrations/20260128120000_backfill_connection_tags.cjs` • `scripts/one-off/connection-tags-migration/*` • `packages/database/lib/backfillConnectionTags.integration.test.ts` • `packages/shared/lib/services/endUser.service.ts` and unit tests </details> --- *This summary was automatically generated by @propel-code-bot* | 7 个月前 | |
fix(database): [nan-1063] migrate to database package (#2236) ## Describe your changes As part of another ticket related to webhooks, I want to make a specific webhook package. I don't want the webhook package to depend on shared but shared contained the database as of now. This PR moves the database connection logic to its own package freeing it from shared and making it easier to use freely in other packages now. Note that the seeders have to stay due to the usage of services. ## Issue ticket number and link Contributes to NAN-1063 ## Checklist before requesting a review (skip if just adding/editing APIs & templates) - [ ] I added tests, otherwise the reason is: - [ ] I added observability, otherwise the reason is: - [ ] I added analytics, otherwise the reason is: | 2 年前 | |
chore(eslint): manual pass over some folders (#3979) ## Changes Didn't want a big PR for import order but it's definitely not going to end without a big push - Import order: manual pass over some folder <!-- This is an auto-generated description by mrge. --> --- ## Summary by mrge Reordered and cleaned up import statements across multiple files to improve code consistency and follow ESLint rules. No functional changes were made. <!-- End of auto-generated description by mrge. --> | 1 年前 | |
feat(nan-5088): scopes remap — accept legacy and new scopes, expand existing keys (#6088) ## Summary Add new fine-grained API key scopes alongside the legacy ones. Routes accept either via `withAnyScope`, so existing keys with legacy scopes keep working. A migration adds the new scopes alongside the legacy ones on existing customer keys (idempotent). Legacy scopes are preserved on keys for rollback safety. Follow-up PRs will: - Remove route mappings for legacy scopes - Strip legacy scope strings from the `scopes` arrays via a final migration ## Scope mapping | Legacy | New | |---|---| | `environment:integrations:write` | `environment:integrations:create` / `update` / `delete` | | `environment:connections:write` | `environment:connections:create` / `update` / `delete` | | `environment:syncs:manage` | `environment:syncs:update` / `variant:create` / `variant:delete` | | `environment:config:read` (for `/environment-variables`) | `environment:variables:read` | | `environment:config:read` (for `/scripts/config`) | `environment:integrations:list_functions` | ## What's included - New scope strings in `@nangohq/types` and `@nangohq/utils` - Route mappings in `routes.public.ts` accept legacy + new via `withAnyScope` - Frontend scope picker shows only the new scopes — legacy scopes are hidden from the picker and excluded from the count column - `stripLegacyScopes` runs in the dashboard save handler so any explicit save on a migrated key drops the legacy entries from the payload (closes the transient window where the API would still authorize via the legacy scope after the user thought they revoked it) - Migration `20260428110800_expand_legacy_api_key_scopes.cjs` — thin wrapper around `expandLegacyApiKeyScopes(knex)` helper - 12 integration tests on the helper directly (no migration runner), break-verified - 38 unit tests on frontend scope-logic ## Rollout plan Three-step rollout designed so each step is independently rollback-safe: 1. **Backup the `customer_keys` table** before deploying this PR. Used as a safety net for the unlikely case that a save during step 1 strips legacy scopes from a key that needs to be rolled back. 2. **Deploy this PR.** Existing keys are expanded in-place (legacy + new scopes side by side). Any newly created key uses only the new scopes. Any save on a migrated key strips the legacy entries from the payload. 3. **Deploy follow-up [PR](https://github.com/NangoHQ/nango/pull/6099)** that removes legacy scope acceptance from public route mappings. After this, only the new scopes authorize requests. 4. **Deploy final PR** that strips legacy scope strings from the `scopes` arrays in the database (cleanup migration). ### Rollback Watch for an increase in `403 Forbidden` responses on the affected endpoints after each step. If one shows up: - After step 2: rollback is safe — legacy scopes are still in the DB on migrated keys, and the previous version of the app accepts them. - After step 3: rollback is safe — legacy scopes are still in the DB and step 3's previous version still accepts them. - After step 4: rollback requires restoring from the backup taken in step 1, since the legacy scope strings are no longer in the DB. Time the deploys with enough spacing (at least one business day) to give the 403 metric room to surface a regression before moving to the next step. ## Test plan - [x] Build passes - [x] 12 helper tests pass (`packages/database/lib/expandLegacyApiKeyScopes.integration.test.ts`) - [x] 38 frontend scope-logic unit tests pass - [x] 49 server scope enforcement integration tests pass (legacy keys still work) - [x] End-to-end manually verified locally: create key with legacy scopes on master → switch to PR branch → migration expands → save in dashboard strips legacy from DB 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> | 3 个月前 | |
fix(db): allow using read-only replica (#3605) ## Changes Contributes to https://linear.app/nango/issue/NAN-2816/use-read-replica-for-big-read-only-queries - Allow to use read-only replica | 1 年前 | |
fix(server): Clamp proxy retry to maximum duration (#7264) <!-- Describe the problem and your solution --> While investigating a spike in server latency at the top of each hour, we discovered long running proxy operations for customer Centralize that was causing requests to stack up at the top of each hour. The root cause seems to be our proxy handler respecting long retries (1 hour+) retrying operations long after the client request will have timed out. There should be a maximum value for retry-after beyond which proxy operations fast fail and return retry information to the caller. <!-- Issue ticket number and link (if applicable) --> [NAN-6759: fix(server): Add maximum retry-after in proxy calls](https://linear.app/nango/issue/NAN-6759/fixserver-add-maximum-retry-after-in-proxy-calls) <!-- Testing instructions (skip if just adding/editing providers) --> To test this I had an agent stand up a mock server temporarily. A quick summary is: 1. Edit `providers.yaml` for a provider like `private-api-bearer` or `unauthenticated` to support retry headers. There might be a provider you could use for this already, but I didn't see it 🤷 : ``` retry: after: - 'retry-after' ``` 2. Standup a mock endpoint that always returns a long retry header based on what you are using from 1 3. Create a new integration and connection for provider from step 1 4. Allow Nango to send traffic to your mock endpoint by setting the following in your `.env`: ``` NANGO_OUTBOUND_URL_POLICY={"blockPrivateIps":false} ``` 5. Proxy a request through Nango connection. It should fail fast with this change rather than hanging for the duration of the retry header you provided from the mock endpoint. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/NangoHQ/nango/pull/7264?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> | 10 天前 | |
chore: upgrade eslint (+ plugins) (#3262) ## Changes Fixes https://linear.app/nango/issue/NAN-2473/upgrade-eslint - Upgrade to eslint 9 and all plugins It's a non-trivial upgrade since eslint chose to have a massive breaking change for their config format. Plus we were behind many months of updates for the plugins. I haven't added/deleted rules, the lines that are changed in this PR are either: previous rules catching more stuff or new rules inside recommended plugins. The only big diff is that I enabled Typescript rules only on ts files which removes some false negative (and makes it slightly faster) | 1 年前 | |
feat(db): migrate Slack connection IDs from env name to env id (#5564) <!-- Describe the problem and your solution --> Adds a knex migration that rewrites Slack notification connection IDs in the admin account from the legacy `account-{uuid}-{envName}` format to the new `account-{uuid}-{envId}` format, scoped to environments with `slack_notifications = true`. Includes a preview SQL helper to capture before/after state for manual rollback (the knex down is a no-op since new and migrated connections are indistinguishable after the dual-lookup deploy). ⚠️ This one has to be merged first !!! https://github.com/NangoHQ/nango/pull/5559 <!-- Issue ticket number and link (if applicable) --> <!-- Testing instructions (skip if just adding/editing providers) --> <!-- Summary by @propel-code-bot --> --- The migration is scoped to admin environments via NANGO_ADMIN_UUID and excludes deleted connections, runs within a transaction with logging of updated row counts, and also adds a SQL helper plus an integration test to validate targeted and non-targeted cases. <details> <summary><strong>Possible Issues</strong></summary> • Comment in `packages/database/lib/migration-helpers/migrateSlackConnectionIdSql.cjs` mentions excluding numeric environment names, but the SQL does not implement this filter. </details> --- *This summary was automatically generated by @propel-code-bot* --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> | 6 个月前 | |
fix(auth): make user emails case-insensitive (#6762) Makes emails case-insensitive. > [!Note] > Before merging and deploying this I will do manual deduplication of existing users that registered with variants of the same email. I have conducted investigation and gathered a list of users to delete. The lowercase email version is not always the correct account to keep. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/NangoHQ/nango/pull/6762?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> | 2 个月前 | |
feat(functions): Production versions of JIT function endpoints (#6214) ## Summary This PR adds the new public Functions API surface for compiling, deploying, and dry-running TypeScript function source code without requiring users to manage a local integration project layout. ### New function endpoints - `POST /functions/compile` - Accepts submitted TypeScript function code and returns bundled JavaScript plus bundle metadata. - Requires the new `environment:functions:compile` API key scope. - `POST /functions/deployments` - Deploys submitted TypeScript function code into an existing integration. - Uses a purpose-bound deploy sandbox token when the sandbox calls back into Nango. - `POST /functions/dryruns` - Starts an asynchronous dry run and returns `202` with an id, status, and `status_url`. - Uses the status values `waiting`, `running`, `success`, and `failed`. - `GET /functions/dryruns/{id}` - Returns the persisted dry run status, output, result, error, duration, and timeout metadata. - `POST /functions/dryruns/{id}/result` - Internal sandbox callback endpoint. - Only accepts purpose-bound sandbox tokens, not customer API keys. The older `/remote-function/*` endpoints remain in place for the existing synchronous compile/deploy/dryrun behavior while sharing the same underlying sandbox clients. ### Async dry run flow `POST /functions/dryruns` creates a `function_dryruns` row, creates a short-lived sandbox API token with dryrun purpose, and starts an E2B sandbox in the background. Inside the sandbox, Nango writes: - the submitted function code - optional input, metadata, checkpoint, and last sync date payloads - a generated Node callback script That callback script compiles the code, runs `nango dryrun`, and posts the final `success` or `failed` payload back to `/functions/dryruns/{id}/result`. The API stores the output/result/error and `GET /functions/dryruns/{id}` exposes the final state. A timeout cron marks stale dry runs as failed if the sandbox never calls back. ### Sandbox package reorganization This moves the sandbox-related business logic out of `server` and `shared` into a new top-level `@nangohq/sandbox` package: - E2B sandbox creation and error mapping - local Docker fallback clients - compile/deploy/dryrun command construction and output parsing - remote function runtime helpers - purpose-bound sandbox API key creation/verification - function dryrun persistence helpers The server package now mostly owns HTTP validation, endpoint orchestration, and response shaping; shared only has enough sandbox key logic required for deserializing it and retrieving the account context from it | 3 个月前 | |
feat(nan-5088): strip legacy scopes from DB + remove dead code (#6179) ## Summary Step 4 (final) of the legacy → fine-grained scope rollout described in #6088. With #6099 in prod, no route accepts the legacy scopes any more, so the strings sitting in \`customer_keys\` are inert. This PR removes them from the DB and rips out every dead code path that existed to deal with them. ## DB - \`20260519120000_strip_legacy_api_key_scopes.cjs\` — single UPDATE that removes the 5 legacy strings from every \`customer_keys.scopes\` array. - Idempotent: \`&&\` overlap filter in the WHERE only touches rows that still carry at least one legacy. - Per the EXPLAIN runs on prod for the expansion migration, this is ~10ms work on ~72K rows. - \`stripLegacyApiKeyScopes\` helper + 10 integration tests covering each legacy scope, idempotency, soft-deleted keys, empty arrays, and unrelated-scope preservation. ## Code - **Types / utils**: removed the 5 legacy entries from the \`ApiKeyScope\` union in \`packages/types\` and \`packages/utils\`. - **Webapp scope picker**: deleted \`LEGACY_SCOPES\` and \`stripLegacyScopes\` from \`scope-logic.ts\`. Removed the 3 call sites in \`ApiKeys.tsx\` (handleSave, countSelectedScopes, hasNoScopes). \`hasNoScopes\` reverts to the plain \`editedScopes.length === 0\` check. - **Tests**: dropped the 3 \`stripLegacyScopes\` unit tests; updated 2 \`scope.middleware.unit.test.ts\` assertions that were probing the wildcard match using legacy strings (the union no longer accepts them). ## Kept - \`expandLegacyApiKeyScopes\` helper + its migration + integration test. They reference legacy strings as plain literals in \`.cjs\` (no type), so they keep working after the union shrinks, and they document what happened on the way through. ## Stack / merge order This PR is stacked on top of #6099 (\`pfreixes/nan-5088-remove-legacy-scopes\`). It must merge after #6099 is deployed and has been live long enough to be confident no rollback is needed (~3–5 days). If a rollback to before #6099 is ever needed after this PR lands, you'd need to restore \`customer_keys\` from the backup taken at step 1 of the rollout — the legacy scope strings will no longer exist on any key. Once #6099 lands on master, retarget the base of this PR to master via the GitHub UI. ## Test plan - [x] 10 helper integration tests pass (\`stripLegacyApiKeyScopes.integration.test.ts\`) - [x] 35 webapp scope-logic unit tests pass (down from 38 after dropping the 3 strip-legacy tests) - [x] 10 server scope.middleware unit tests pass (updated 2 assertions) - [x] \`npm run ts-build\` clean across the workspace - [ ] CI scope enforcement integration tests pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com> | 3 个月前 | |
feat: move webhook_url override to top-level field. (#6778) > This is a breaking change with low surface impact After a lot of discussion we believe that `connection_config` was not the right place for `webhook_url`. We are trying to move quick while it's still not massively adopted (currently 2 accounts, using it incorrectly as the main `webhook_url` setting). This moves the setting to a top-level property on the connection called `webhook_url_override`. I also considered moving it to `overrides` on the connect_session, but the `overrides` object is keyed by integration, which doesn't really make sense for `webhook_url`. So I'm proposing this top-level alternative instead. Please look at the docs updates to understand the user-level change. --------- Co-authored-by: Cursor <cursoragent@cursor.com> | 1 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 个月前 | ||
| 2 天前 | ||
| 7 个月前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 3 个月前 | ||
| 1 年前 | ||
| 10 天前 | ||
| 1 年前 | ||
| 6 个月前 | ||
| 2 个月前 | ||
| 3 个月前 | ||
| 3 个月前 | ||
| 1 个月前 |