Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 6 additions & 11 deletions .agents/skills/ship/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,19 +47,14 @@ When the user runs `/ship`:
- Run `/db-migrate` to review the migration for zero-downtime safety (expand/contract phasing, backward-compatibility with the deployed app version).
- `(cd packages/db && bunx drizzle-kit generate && git status --porcelain ./migrations)` must print nothing (CI's schema/migration sync step).
- `bun run check:migrations origin/staging` must pass (staging is the PR base). Do not silence a flagged statement with a `-- migration-safe:` annotation unless `/db-migrate` confirmed the old code no longer depends on it; otherwise split the destructive change into a later deploy.
6. **Run pre-ship checks** from the repo root before staging. This has two phases: first **regenerate** every committed artifact so generated files never drift into a CI failure (this is what catches things like `agent-stream-docs` going stale after a `models.ts` edit), then run the **full audit suite** CI's `lint` job enforces. Both phases parallelize — but only across commands that write **disjoint** outputs — and a bare `wait` swallows child exit codes, so both phases below explicitly collect each job's status and abort ship if any failed.
6. **Run pre-ship checks** from the repo root before staging. This has two phases: first **regenerate** every committed artifact so generated files never drift into a CI failure (this is what catches things like `agent-stream-docs` going stale after a `models.ts` edit), then run the **full audit suite** CI's `lint` job enforces. Both phases abort ship if any generator or audit fails: their runners (`scripts/generate-all.ts`, `scripts/run-audits.ts`) collect each job's exit code and exit non-zero, so never replace them with a backgrounded loop and a bare `wait`, which swallows child exit codes.

**Phase A — regenerate the always-in-repo committed artifacts (parallel), then let step 7 stage whatever changed.** Regenerate only the generators whose inputs live entirely in this repo and that any ordinary code change can drift — `generate:agent-stream-docs` (derives from the provider model registry), `generate:docs-manifest` (derives from docs page paths), `generate:tool-metadata` (derives from the tool registry), and `skills:sync` (derives from `.agents/skills/**`). They write disjoint outputs (`apps/docs/…/agent.mdx`, `apps/sim/lib/mothership/generated/docs-manifest.ts`, `apps/sim/tools/generated/*`, and `.claude/skills` links), so they parallelize safely, and each is idempotent (a no-op when already in sync):
**Phase A — regenerate the always-in-repo committed artifacts, then let step 7 stage whatever changed.** Run the umbrella generator. It derives its list from package.json (every `generate:<x>` that has a matching `check:<x>`, plus `skills:sync`; see the header of `scripts/generate-all.ts`), so it covers every repo-local generator an ordinary code change can drift (a `models.ts` edit drifts `agent-stream-docs` and the SDK fixture catalog, a tool change drifts `tool-metadata`), runs them in the order their inputs require, and each is idempotent (a no-op when already in sync):
```bash
rm -f /tmp/ship-gen-results
for g in generate:agent-stream-docs generate:docs-manifest generate:tool-metadata skills:sync; do
( bun run "$g" >"/tmp/ship-gen-${g//:/-}.log" 2>&1; echo "$? $g" >>/tmp/ship-gen-results ) &
done
wait
# any non-zero line is a FAILED generator — read /tmp/ship-gen-<name>.log and fix before shipping;
# a silently-failed generate leaves a stale artifact that Phase B / CI then rejects. Keep the
# `exit 1`: it is what makes the block's own status non-zero so a caller actually stops.
if grep -vE '^0 ' /tmp/ship-gen-results; then echo "❌ generator(s) failed — do not ship"; exit 1; fi
# A non-zero exit means a generator FAILED: the summary names it and its output is above. Fix it
# before shipping; a silently-failed generate leaves a stale artifact that Phase B / CI then
# rejects. Keep the `exit 1`: it makes the block's own status non-zero so a caller actually stops.
bun run generate || { echo "❌ generator(s) failed — do not ship"; exit 1; }
echo "✅ artifacts regenerated"
```
Then `git status --short` to see what regenerated — those files must be staged in step 7 alongside your own changes.
Expand Down
25 changes: 1 addition & 24 deletions apps/sim/lib/workflows/code/real-catalog.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* workflows against the types generated from the whole registry with the workspace's TypeScript 7
* compiler.
*/
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import path from 'node:path'
import { createBlock } from '@sim/testing/factories/block.factory'
Expand Down Expand Up @@ -32,9 +32,7 @@ import { afterAll, describe, expect, it, vi } from 'vitest'
vi.unmock('@/blocks/registry')
vi.unmock('@/tools/metadata-outputs')

import { projectBlockCatalog } from '@/lib/workflows/code/catalog/build'
import { registryCatalog as catalog } from '@/lib/workflows/code/catalog/registry-catalog.test-helpers'
import { registryTrigger } from '@/lib/workflows/code/catalog/registry-sources'
import { generateRegistryCorpus } from '@/lib/workflows/code/corpus'
import { createMaterializeHooks } from '@/lib/workflows/code/materialize-hooks'
import {
Expand All @@ -44,7 +42,6 @@ import {
} from '@/lib/workflows/code/sdk-typecheck.test-helpers'
import { BUILTIN_SKILLS } from '@/lib/workflows/skills/builtin-skills'
import { getBlockRegistry } from '@/blocks/registry'
import { SDK_NAMES } from '@/blocks/sdk-names.generated'

const registryBlocks = Object.values(getBlockRegistry())

Expand Down Expand Up @@ -82,26 +79,6 @@ const COOKBOOK_WORKSPACE: WorkspaceManifest = {
}

describe('canonicalize and codegen over the real block catalog', () => {
it('keeps the SDK fixture catalog equal to the real projection of its blocks', () => {
const fixturePath = path.join(SDK_ROOT, 'test/fixtures/catalog.json')
const fixture = JSON.parse(readFileSync(fixturePath, 'utf8')) as { blocks: object }
const types = new Set(Object.keys(fixture.blocks))
const projected = {
...projectBlockCatalog({
blocks: registryBlocks.filter((block) => types.has(block.type)),
trigger: registryTrigger,
sdkNames: SDK_NAMES,
}),
hash: 'fixture',
}
const text = `${JSON.stringify(projected, null, 2)}\n`
if (process.env.UPDATE_SDK_FIXTURE_CATALOG === '1') writeFileSync(fixturePath, text)
expect(
JSON.parse(readFileSync(fixturePath, 'utf8')),
'Regenerate with UPDATE_SDK_FIXTURE_CATALOG=1 bun run test lib/workflows/code/real-catalog.test.ts'
).toEqual(JSON.parse(text))
})

it("never freezes the host's timezone into the catalog as the schedule default", () => {
const timezone = catalogBlock(catalog, 'schedule')?.subBlocks.find(
(sub) => sub.id === 'timezone'
Expand Down
59 changes: 59 additions & 0 deletions apps/sim/scripts/generate-sdk-fixture-catalog.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
#!/usr/bin/env bun
/**
* Maintains `packages/workflow-sdk/test/fixtures/catalog.json`, the block catalog the SDK and CLI
* tests compile against.
*
* The fixture is not hand-written: it is the real `projectBlockCatalog` projection of a subset of
* registry blocks, so a test over it exercises the shapes production emits. The subset is the set
* of block types already in the fixture; add a block by adding its type key to `blocks` and
* regenerating. The hash is pinned to `'fixture'` so the file changes only when a projected block
* does.
*
* Any change to a projected block, its tools' outputs, its SDK names, or a model list it reads
* drifts this file, which is why `--check` runs in `check:audits` instead of leaving the drift for
* a test shard to find.
*
* Usage:
* bun run apps/sim/scripts/generate-sdk-fixture-catalog.ts # rewrite the fixture
* bun run apps/sim/scripts/generate-sdk-fixture-catalog.ts --check # fail if it is stale
*/
import { readFile, writeFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import { projectBlockCatalog } from '@/lib/workflows/code/catalog/build'
import { registryTrigger } from '@/lib/workflows/code/catalog/registry-sources'
import { getBlockRegistry } from '@/blocks/registry'
import { SDK_NAMES } from '@/blocks/sdk-names.generated'

const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
const FIXTURE_PATH = resolve(
SCRIPT_DIR,
'../../../packages/workflow-sdk/test/fixtures/catalog.json'
)
const FIXTURE_LABEL = 'packages/workflow-sdk/test/fixtures/catalog.json'

const checkOnly = process.argv.includes('--check')
const current = await readFile(FIXTURE_PATH, 'utf8')
const types = new Set(Object.keys((JSON.parse(current) as { blocks: object }).blocks))

const blocks = Object.values(getBlockRegistry()).filter((block) => types.has(block.type))
const projected = {
...projectBlockCatalog({ blocks, trigger: registryTrigger, sdkNames: SDK_NAMES }),
hash: 'fixture',
}
const text = `${JSON.stringify(projected, null, 2)}\n`

if (checkOnly) {
if (current !== text) {
console.error(
`${FIXTURE_LABEL} is stale: it no longer matches the registry projection of its ${types.size} blocks. Run \`bun run generate:sdk-fixture-catalog\` and commit the result.`
)
process.exit(1)
}
console.log(`${FIXTURE_LABEL} matches the registry projection of its ${types.size} blocks.`)
} else if (current === text) {
console.log(`${FIXTURE_LABEL} is already up to date (${types.size} blocks).`)
} else {
await writeFile(FIXTURE_PATH, text)
console.log(`Regenerated ${FIXTURE_LABEL} (${types.size} blocks).`)
}
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@
"check:sealed-columns": "bun run scripts/check-sealed-columns.ts",
"check:schema-mock": "bun run scripts/generate-schema-mock.ts --check",
"check:audits": "bun run scripts/run-audits.ts",
"generate": "bun run scripts/generate-all.ts",
"check:skills": "bun run scripts/sync-skills.ts --check",
"check:guidance-refs": "bun run scripts/check-guidance-refs.ts",
"check:desktop-bridge": "bun run scripts/check-desktop-bridge-contract.ts --check",
Expand All @@ -102,6 +103,8 @@
"check:tool-metadata": "bun run scripts/sync-tool-metadata.ts --check",
"generate:sdk-names": "bun run apps/sim/scripts/generate-sdk-names.ts",
"check:sdk-names": "bun run apps/sim/scripts/generate-sdk-names.ts --check",
"generate:sdk-fixture-catalog": "bun run apps/sim/scripts/generate-sdk-fixture-catalog.ts",
"check:sdk-fixture-catalog": "bun run apps/sim/scripts/generate-sdk-fixture-catalog.ts --check",
"check:sdk-names-frozen": "bun run scripts/check-sdk-names-frozen.ts",
"generate:deployment-config": "bun run scripts/generate-deployment-config.ts",
"check:deployment-config": "bun run scripts/generate-deployment-config.ts --check",
Expand Down
Loading
Loading