Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/backup-restore-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,24 @@ jobs:
- name: Install PlanetScale CLI
uses: planetscale/setup-pscale-action@b6a50ee45b4b24944e1d8de6e57b3a5f6476a1af # v1

# A short-lived service token from the `maintenance` OIDC trust, which grants
# only the drill's accesses (docs/backup-and-recovery.md). The org slug comes
# masked from Infisical, since the run logs are public.
- name: PlanetScale credentials (OIDC)
id: planetscale
uses: planetscale/auth-action@aa8f73f656f858eb798916ce2fe9769ab1ba0d76 # v1.0.0
with:
organization: ${{ env.PLANETSCALE_ORG }}
oidc-trust: ${{ vars.PLANETSCALE_OIDC_TRUST_ID }}

# The report goes to a non-hidden directory: upload-artifact skips
# dot-directories by default, which silently dropped the first run's
# report from `packages/db/.restore-test/`.
- name: Restore the newest backup and verify it
env:
RESTORE_TEST_REPORT_DIR: ${{ runner.temp }}/backup-restore-test
PLANETSCALE_SERVICE_TOKEN_ID: ${{ steps.planetscale.outputs.service-token-id }}
PLANETSCALE_SERVICE_TOKEN: ${{ steps.planetscale.outputs.service-token }}
run: bun run --cwd packages/db db:restore-test

# Uploaded even when the drill fails: a failed report is the evidence
Expand Down
13 changes: 11 additions & 2 deletions .github/workflows/deploy-prd-instance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,15 @@ jobs:
aws-role-arn: ${{ vars.AWS_DEPLOY_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}

# Schema migrations run in this deploy (`Planetscale.PostgresBranch`), using the
# PLANETSCALE_* credentials from Infisical.
# Schema migrations run in this deploy (`Planetscale.PostgresBranch`). Its PlanetScale
# service token is short-lived, minted over OIDC by the trust for this environment;
# the org slug comes masked from Infisical, since the run logs are public.
- name: PlanetScale credentials (OIDC)
id: planetscale
uses: planetscale/auth-action@aa8f73f656f858eb798916ce2fe9769ab1ba0d76 # v1.0.0
with:
organization: ${{ env.PLANETSCALE_ORGANIZATION }}
oidc-trust: ${{ vars.PLANETSCALE_OIDC_TRUST_ID }}

# AWS_ACCOUNT_ID skips alchemy's STS account lookup, which self-deadlocks under CI=true (#378).
# One pass, no retry: the stack validates its own ACM certificates.
Expand All @@ -68,6 +75,8 @@ jobs:
run: bun run alchemy:deploy:prd
env:
AWS_ACCOUNT_ID: ${{ steps.setup.outputs.aws-account-id }}
PLANETSCALE_API_TOKEN_ID: ${{ steps.planetscale.outputs.service-token-id }}
PLANETSCALE_API_TOKEN: ${{ steps.planetscale.outputs.service-token }}

# alchemy isolates per-Worker failures, so a deploy can leave prod on two commits.
# Runs on failure too, to name the stale Worker. Other Workers are covered by the
Expand Down
30 changes: 23 additions & 7 deletions docs/backup-and-recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,14 +74,14 @@ a recovery incident.
`branch show`, before anything is created.
- **In CI:** the workflow runs only on its schedule or via
`gh workflow run backup-restore-test.yml` after merge. A PR's CI does not run it. The first
dispatch validates the service token. On the database it needs `read_branch`,
dispatch validates the OIDC trust. On the database it needs `read_branch`,
`create_branch`, `delete_branch`, `connect_branch`, `delete_branch_password`, `read_backups`
and `restore_production_branch_backup`. Beyond the restore access these are
development-branch accesses; it never needs `delete_production_branch`. A token missing
development-branch accesses; it never needs `delete_production_branch`. A trust missing
`read_backups` fails at `backup list`. One missing the restore access fails at
`branch create`. Both fail before anything is created. One missing the delete accesses
restores fine and then leaves the branch behind, which the drill reports as a failed check.
Only the token's creator or an organization administrator can change its accesses.
Only an organization administrator can change a trust's accesses.
- **Why the restore is `--cluster-size PS_DEV`:** a restore without an explicit size becomes a
PS-10 branch flagged `production: true`, which only `delete_production_branch` can remove.
PlanetScale cannot demote a Postgres restore branch afterwards. A PS_DEV restore is a
Expand All @@ -90,10 +90,26 @@ a recovery incident.
### Schedule

[`backup-restore-test.yml`](../.github/workflows/backup-restore-test.yml) runs the drill at 06:23
UTC on the first day of every quarter and on `workflow_dispatch`. It authenticates the `pscale`
CLI with the PlanetScale service token from Infisical (`PLANETSCALE_SERVICE_TOKEN_ID` / `_TOKEN` /
`PLANETSCALE_ORG`). The preview-orphan sweep uses the same token, which already has branch create
and delete on the `maple` database.
UTC on the first day of every quarter and on `workflow_dispatch`. No PlanetScale credential is
stored for it: `planetscale/auth-action` trades the job's GitHub OIDC token for a service token
that lives 45 minutes (the job timeout), and the step hands it to `pscale` as
`PLANETSCALE_SERVICE_TOKEN_ID` / `_TOKEN`. The org slug is `PLANETSCALE_ORG` from Infisical `dev`,
masked in the public run logs. The trust's ID is the `PLANETSCALE_OIDC_TRUST_ID` variable on the
`maintenance` GitHub environment, and the trust accepts only that environment of this repository:

```bash
pscale oidc-trust create --org <org> --name "github: backup restore drill" \
--provider github_actions --max-ttl 2700 \
--condition repository_id=<repo id> --condition environment=maintenance \
--database-access maple:read_branch --database-access maple:create_branch \
--database-access maple:delete_branch --database-access maple:connect_branch \
--database-access maple:delete_branch_password --database-access maple:read_backups \
--database-access maple:restore_production_branch_backup
```

`gh api repos/MapleTechLabs/maple --jq .id` prints the repository id. Check the claim names with
`pscale oidc-trust providers show github_actions --org <org>`; the provider may require more
conditions (such as `repository_owner_id`).

Point-in-time recovery is the other restore path PlanetScale offers: any timestamp inside the
retention window, from the dashboard or `pscale branch create --from main --restore-point <iso>`
Expand Down
21 changes: 18 additions & 3 deletions docs/persistence.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,24 @@ mints one role per consumer on `maple-eu` and a Hyperdrive config on each role's
the Workers bind them from their props. No dashboard config and no hand-minted role exist there
(its `resolveMapleProfile` database mode is `"declared"`); the US prd keeps its dashboard-managed configs, bound by id.

The deploy reads `PLANETSCALE_API_TOKEN_ID` / `PLANETSCALE_API_TOKEN` /
`PLANETSCALE_ORGANIZATION` from the instance's Infisical environment. `bun dev` leaves the
PlanetScale provider out.
The deploy stores no PlanetScale credential. `deploy-prd-instance.yml` trades the job's GitHub
OIDC token for a service token through `planetscale/auth-action` and hands it to alchemy as
`PLANETSCALE_API_TOKEN_ID` / `PLANETSCALE_API_TOKEN`. The token lives 45 minutes, the job
timeout, since alchemy resolves it once and migrations run late in the deploy.
`PLANETSCALE_ORGANIZATION` still comes from the instance's Infisical environment, masked in the
public run logs. Each instance has its own trust, conditioned on its GitHub environment
(`production`, `production-eu`), whose ID is that environment's `PLANETSCALE_OIDC_TRUST_ID`
variable. Give a trust the accesses the stored token had, which `pscale service-token show-access
<id> --org <org>` lists:

```bash
pscale oidc-trust create --org <org> --name "github: prd deploy" \
--provider github_actions --max-ttl 2700 \
--condition repository_id=<repo id> --condition environment=production \
--database-access <database>:<access> ...
```

`bun dev` leaves the PlanetScale provider out.

PGlite applies the same bundled migrations while its layer is built. The test harness caches a
fresh migrated PGlite snapshot and restores it per test, so integration tests exercise the
Expand Down
Loading