Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ The `changes` job runs Turborepo 2.11.10's `affectedTasks` query before installi

A PR compares the checked-out GitHub merge commit with its first parent: the actual target revision tested by that merge. The PR event's base SHA can become stale when the target advances. Scoping requires a matching GitHub merge ref and SHA, exactly two merge parents, and a usable target parent; otherwise every check runs. This works with the existing depth-two checkout without fetching older ancestry. Main/staging pushes execute all checks; the parent `ci` workflow skips checks on `dev` pushes, while PRs targeting `dev` still run checks. A manual `checks` run executes all checks unless supplied a full base commit SHA to reproduce a scoped plan. Missing history, an empty diff, unknown files outside the workspace graph, root script/CI changes, or a planner error run every check. Missing selection outputs fail the aggregate gate. The gate also rejects an omitted dependency, any failed/cancelled job, and any skip that its explicit selection flag did not authorize.

Changes exclusively to the reviewed library-content or utils-enforcement audit scripts and their adjacent tests run the always-on lint/audit job and a separate job for the complete root script test suite. The lint job defaults to Blacksmith; the root test job uses one GitHub runner. They do not change application runtime. Mixed changes, deleted inputs, unknown scripts, and CI or generator changes retain full fallback. The attribution artifact records this explicit audit-only owner.

Dedicated docs verification uses the same planner, comparing pushes with their previous commit and PRs with the tested merge's first parent. Site acceptance follows `docs#build` and its transitive package dependencies. The app's docs-consumer task depends on server runtime and that docs build: backend-only changes run the platform consumer against the docs app, while site changes also run corpus and browser checks. Standalone package and unit-only edits skip those production jobs. The required `docs HTTP` gate verifies explicit site/platform selections before accepting any skip.

Docs production builds run once and publish an artifact to isolated HTTP, corpus, Chromium, and WebKit jobs. Source checks keep their own checkout because generator verification mutates inputs. Content caches use separate lint/source/build namespaces; generation validates every restore, repairs missing declarations, and removes obsolete modules. The Next build cache keeps its supported incremental behavior. Content and Next caches have size limits, and Next cache files expire after seven days; successful content eviction happens after all consumers finish.
Expand Down
63 changes: 63 additions & 0 deletions scripts/ci-changes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,69 @@ afterEach(() => {
for (const repo of fixtures.splice(0)) rmSync(repo, { recursive: true, force: true })
})
describe('CI task attribution through real Turborepo', () => {
describe('reviewed root audits', () => {
it.each([
{ files: ['scripts/check-library-content.ts', 'scripts/check-library-content.test.ts'] },
{ files: ['scripts/check-utils-enforcement.test.ts'] },
])('runs audits and script tests without app infrastructure for $files', ({ files }) => {
const plan = classifyFiles(files)
expect(plan.fallback).toBeUndefined()
expect(plan.flags).toEqual({
docs: false,
docs_site: false,
docs_platform: false,
integration: false,
http: false,
production: false,
e2e_build: false,
desktop_live: false,
test: true,
test_app: false,
build: false,
})
expect(groups(plan)).toEqual([])
expect(plan.testPackages).toEqual([])
expect(plan.typecheckPackages).toEqual([])
expect(plan.docsGroups).toEqual([])
})
it.each(['apps/sim/lib/server.ts', 'scripts/check-example.ts', 'AGENTS.md'])(
'runs everything when a reviewed audit changes alongside %s',
(file) => {
const plan = classifyFiles(['scripts/check-library-content.ts', file])
expect(plan.fallback).toBeDefined()
expect(Object.values(plan.flags).every(Boolean)).toBe(true)
}
)
it.each(['delete', 'move'])('runs everything for a reviewed audit %s', (operation) => {
const { repo } = fixture()
const file = 'scripts/check-library-content.ts'
write(repo, file, 'export const audit = true\n')
git(repo, 'add', '-A')
git(repo, 'commit', '--quiet', '-m', 'audit owner')
const base = git(repo, 'rev-parse', 'HEAD')
if (operation === 'delete') git(repo, 'rm', file)
else git(repo, 'mv', file, 'scripts/unknown-audit.ts')
git(repo, 'commit', '--quiet', '-am', 'remove audit owner')
const plan = classify(repo, base)
expect(plan.fallback).toBeDefined()
expect(Object.values(plan.flags).every(Boolean)).toBe(true)
})
it('requires PR merge provenance even for a reviewed audit', () => {
const { repo, base, environment } = pullRequestFixture(['scripts/check-library-content.ts'])
const plan = classify(repo, base, { ...environment, GITHUB_REF: 'refs/heads/staging' })
expect(plan.fallback).toBeDefined()
expect(Object.values(plan.flags).every(Boolean)).toBe(true)
})
it('keeps unscoped pushes full even for a reviewed audit', () => {
const { repo } = fixture()
write(repo, 'scripts/check-library-content.ts', 'export const audit = true\n')
git(repo, 'add', '-A')
git(repo, 'commit', '--quiet', '-m', 'audit change')
const plan = classify(repo, '', { GITHUB_EVENT_NAME: 'push' })
expect(plan.fallback).toBeDefined()
expect(Object.values(plan.flags).every(Boolean)).toBe(true)
})
})
it('scopes a shallow multi-commit PR against its tested target after the event base goes stale', () => {
const files = ['packages/sim-setup/src/index.ts', 'packages/sim-setup/src/second.ts']
const { repo, base, target, environment } = pullRequestFixture(files)
Expand Down
93 changes: 54 additions & 39 deletions scripts/ci-plan.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { spawnSync } from 'node:child_process'
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join, resolve } from 'node:path'

Expand All @@ -23,6 +23,13 @@ interface QueryResult {
docsSite: { items: { fullName: string }[] }
}
}
/** These audit-only inputs belong to lint and test:scripts, rather than application runtime. */
const ROOT_AUDIT_INPUTS = new Set([
'scripts/check-library-content.ts',
'scripts/check-library-content.test.ts',
'scripts/check-utils-enforcement.ts',
'scripts/check-utils-enforcement.test.ts',
])
const root = process.cwd()
const suites: { http: Suite[]; production: Suite[] } = JSON.parse(
readFileSync(resolve(root, '.github/ci-suites.json'), 'utf8')
Expand All @@ -31,6 +38,7 @@ const sourceBase = process.argv[2]
let files: string[] = []
let tasks: ChangedTask[] = []
let docsSiteChanged = false
let rootAuditOnly = false
let base = sourceBase
let fallback: string | undefined

Expand Down Expand Up @@ -66,42 +74,47 @@ try {
.split('\0')
.filter(Boolean)
if (files.length === 0) throw new Error('Empty diff; verify the complete suite')
if (files.some((file) => /^(\.github\/|scripts\/)/.test(file)))
throw new Error('CI orchestration or root scripts changed')
const temp = mkdtempSync(join(tmpdir(), 'sim-ci-plan-'))
let result: QueryResult
try {
const variables = join(temp, 'variables.json')
writeFileSync(variables, JSON.stringify({ base, head: 'HEAD' }))
result = JSON.parse(
command('turbo', ['query', '.github/scripts/ci-plan.gql', '--variables', variables])
)
} finally {
rmSync(temp, { recursive: true, force: true })
}
if (
result.errors?.length ||
!Array.isArray(result.data?.affectedTasks?.items) ||
!Array.isArray(result.data?.docsSite?.items) ||
!Array.isArray(result.data?.packages?.items)
)
throw new Error('Turborepo did not return a complete task plan')
const workspacePaths = result.data.packages.items
.filter((pkg) => pkg.name !== '//')
.map((pkg) => `${pkg.path}/`)
const guidance = /^(\.agents|\.claude|\.cursor)\/|(^|\/)(AGENTS|CLAUDE)\.md$|^README\.md$/
const unowned = files.find(
(file) => !guidance.test(file) && !workspacePaths.some((path) => file.startsWith(path))
rootAuditOnly = files.every(
(file) => ROOT_AUDIT_INPUTS.has(file) && existsSync(resolve(root, file))
)
if (unowned) throw new Error(`Changed file outside the workspace graph: ${unowned}`)
tasks = result.data.affectedTasks.items
if (
tasks.some(
(task) => typeof task.fullName !== 'string' || typeof task.package?.name !== 'string'
if (!rootAuditOnly) {
if (files.some((file) => /^(\.github\/|scripts\/)/.test(file)))
throw new Error('CI orchestration or root scripts changed')
const temp = mkdtempSync(join(tmpdir(), 'sim-ci-plan-'))
let result: QueryResult
try {
const variables = join(temp, 'variables.json')
writeFileSync(variables, JSON.stringify({ base, head: 'HEAD' }))
result = JSON.parse(
command('turbo', ['query', '.github/scripts/ci-plan.gql', '--variables', variables])
)
} finally {
rmSync(temp, { recursive: true, force: true })
}
if (
result.errors?.length ||
!Array.isArray(result.data?.affectedTasks?.items) ||
!Array.isArray(result.data?.docsSite?.items) ||
!Array.isArray(result.data?.packages?.items)
)
)
throw new Error('Malformed task attribution')
docsSiteChanged = result.data.docsSite.items.some((task) => task.fullName === 'docs#build')
throw new Error('Turborepo did not return a complete task plan')
const workspacePaths = result.data.packages.items
.filter((pkg) => pkg.name !== '//')
.map((pkg) => `${pkg.path}/`)
const guidance = /^(\.agents|\.claude|\.cursor)\/|(^|\/)(AGENTS|CLAUDE)\.md$|^README\.md$/
const unowned = files.find(
(file) => !guidance.test(file) && !workspacePaths.some((path) => file.startsWith(path))
)
if (unowned) throw new Error(`Changed file outside the workspace graph: ${unowned}`)
tasks = result.data.affectedTasks.items
if (
tasks.some(
(task) => typeof task.fullName !== 'string' || typeof task.package?.name !== 'string'
)
)
throw new Error('Malformed task attribution')
docsSiteChanged = result.data.docsSite.items.some((task) => task.fullName === 'docs#build')
}
} catch (error) {
fallback = String(error)
}
Expand Down Expand Up @@ -141,7 +154,7 @@ const flags = {
production: production.length > 0,
e2e_build: http.length > 0 || production.length > 0 || desktopLive,
desktop_live: desktopLive,
test: appTest || packages.length > 0,
test: rootAuditOnly || appTest || packages.length > 0,
test_app: appTest,
build: affected('build'),
}
Expand All @@ -159,6 +172,7 @@ const report = {
head: command('git', ['rev-parse', 'HEAD']),
files,
fallback,
rootAuditOnly,
tasks,
flags,
http,
Expand All @@ -169,9 +183,10 @@ const report = {
}
if (process.env.CI_PLAN_PATH)
writeFileSync(process.env.CI_PLAN_PATH, `${JSON.stringify(report, null, 2)}\n`)
process.stderr.write(
`${fallback ? `Full CI: ${fallback}` : `Affected CI: ${tasks.map((task) => task.fullName).join(', ') || 'no executable tasks'}`}\n`
)
const attribution = rootAuditOnly
? 'reviewed root audits and script tests'
: tasks.map((task) => task.fullName).join(', ') || 'no executable tasks'
process.stderr.write(`${fallback ? `Full CI: ${fallback}` : `Affected CI: ${attribution}`}\n`)
process.stdout.write(
Object.entries(outputs)
.map(([key, value]) => `${key}=${value}\n`)
Expand Down
Loading