Add IBM Equal Access Accessibility Checker to the Cypress a11y suite - #3357
Merged
Merged
Conversation
Wires cypress-accessibility-checker alongside the existing cypress-axe checks in the admin and form-preview a11y specs, via a new cy.checkIbmAccessibility(label) command. Runs in report-only mode (assertCompliance(false)) for now, since the existing markup hasn't been triaged against this rule set - see follow-up Strategy11/formidable-forms#3356. Fix Strategy11/formidable-pro#6682
vivi-the-going-merry
Bot
added
the
run e2e tests
Contributor
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
 |
|
Overall Grade |
Security Reliability Complexity Hygiene |
Code Review Summary
| Analyzer | Status | Updated (UTC) | Details |
|---|---|---|---|
| PHP |  |
Sep 17, 2026 3:11p.m. | Review ↗ |
| JavaScript | |
Sep 17, 2026 3:11p.m. | Review ↗ |
Important
AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.
The .then() callback invoked cy.task() (queued, async) and then returned report synchronously in the same branch - Cypress treats that as an error. That aborted the scan, and because this suite runs with one retry enabled, the retry re-invoked getCompliance() with the same label, which the IBM checker rejects as a label reused within the same process/session. Chain the logging tasks and resolve back to report at the end instead of returning it alongside already-queued commands.
Contributor
|
Tests are failing. |
Contributor
Author
|
That log is from the original commit (a132cbe) - already fixed in 0bea79b (pushed before this comment): checkIbmAccessibility mixed a queued cy.task() with a synchronous return inside the same .then() branch, which Cypress rejects. That failed every test using the command on its first attempt, and the configured retry then hit labelNotUnique against the label each first attempt had already registered - same root cause behind all 11 failures pasted above, not 11 separate bugs. Latest run (35252053900): the a11y specs pass; the one remaining failure (shard 1, Form Templates search) is unrelated - a template list search assertion, nothing to do with this suite. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was broken
The Lite Cypress suite only ran
cypress-axe(axe-core) a11y checks. IBM's Equal Access checker uses a different engine and catches an overlapping-but-distinct rule set — requested as additive coverage, not a replacement.What changed
cypress-accessibility-checkeras a dev dependency, registered its Cypress plugin/task incypress.config.js, and imported it in the support file..achecker.yml(IBM_Accessibility policy, fail onviolationonly).cy.checkIbmAccessibility(label)command (tests/cypress/support/commands.js) that scans the current page and logs any non-passing results via the existinglog/tabletasks, the same pattern the suite already uses for axe violations.cy.checkA11y(...)site intests/cypress/e2e/admin-a11y.cy.jsandform-preview-a11y.cy.js(12 pages total).Why report-only for now
The existing admin/preview markup hasn't been triaged against IBM's rule set yet, and per the issue's own ask, currently-failing checks get an explicit exception rather than blocking on a full remediation pass.
checkIbmAccessibilitycallsassertCompliance(false)— it scans and logs every run but doesn't fail CI. Follow-up to triage the logged violations and flip on enforcement: Strategy11/formidable-pro#6687.How verified
No local Cypress run (this repo has real CI —
.github/workflows/cypress.yml, gated on therun e2e testslabel). Pushed, then labeled this PR to trigger it; will check the run and comment with the result.node --checkon every touched.jsfile,python3 -c "import yaml; yaml.safe_load(...)"on.achecker.yml, and a JSON diff verification thatpackage-lock.json's only changes are the 12 new packagescypress-accessibility-checkeractually needs plus its own line indevDependencies(regenerated inside a Node 22 container to match CI, then grafted minimally rather than committing the full re-resolve — a from-scratchnpm installon this lockfile produces a large unrelated diff from an npm-version mismatch).Closes Strategy11/formidable-pro#6682