Skip to content

Enforce Web Platform Test capabilities in CI - #632

Open
andrewiggins wants to merge 7 commits into
wpt-runner-interactivefrom
wpt-runner-capability-ci
Open

Enforce Web Platform Test capabilities in CI#632
andrewiggins wants to merge 7 commits into
wpt-runner-interactivefrom
wpt-runner-capability-ci

Conversation

@andrewiggins

@andrewiggins andrewiggins commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR is 3 of 3 (#630, #631) that adds a Web Platform Test runner to this repo, so we can validate our polyfills spec compliance against that test suite.

This PR adds a CLI to run the webpage from the previous PR in playwright. It also adds a way to specify which tests in WPT are expected to pass/fail, and why (if failure). So overtime we can document behaviors we support, ones we don't support yet, and which ones we explicitly don't want to support.

Specifically, this PR:

  • add a canonical TSV inventory of supported and deferred WPT cases
  • reject supported failures, missing results, unlisted results, duplicates, harness errors, worker errors, and timeouts
  • report passing deferred cases as promotion candidates
  • add a Playwright CLI that reuses one control page while creating a fresh worker per file
  • add a dedicated cached WPT CI job

Stacked dependency

This PR is stacked on wpt-runner-interactive, which provides secure serving and isolated worker execution.

Executable outcome

Run pnpm test:wpt to execute every classified path and enforce the complete capability table. Use explicit paths for exploratory execution and --capabilities to enforce selected classified paths.

Validation

  • pnpm run lint
  • pnpm run type-check
  • pnpm exec vitest run
  • pnpm run test:wpt
  • classified three-case smoke execution

Changeset

An empty changeset records that this private test infrastructure releases no public package.

@andrewiggins
andrewiggins force-pushed the wpt-runner-capability-ci branch from c79bf55 to 7dadbe6 Compare August 21, 2026 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant