Record the commit the job actually checked out, not GITHUB_SHA by TR1LON · Pull Request #255 · ForAllSecure/mcode-action

TR1LON · 2026-09-21T14:48:26Z

Ticket: PE-8976 (assumed: the commit-id half of the pair; PE-8975 is the org stamps)

Problem

The run's --revision is taken from GITHUB_SHA, which is fixed when the workflow run starts. A job that checks out a different ref labels the Mayhem run with a commit it did not build. That happens for every reusable workflow called with a freshly rebased commit, for workflow_run handlers, and for any explicit ref: on actions/checkout.

Concrete case, savantenvs/delta on 2026-09-20 (Actions run 35507899158, a scheduled sync.yml):

	commit
`GITHUB_SHA` (sync cron started from)	`4b79bc9`
`sync.yml` rebased `mayhem` to, and passed as `ref:`	`6ac2067`
"fuzz / build commit image" resolved and built	`commit-6ac2067`
"fuzz / fuzz mayhem/Mayhemfile" checked out	`6ac2067`
Mayhem run #4 `revision`	`4b79bc9`

The rlenv gate compares a run's revision with the fork's mayhem HEAD, so it read this healthy run as stale and demoted the repo. 109 repos in that fleet hit the same thing this week; in a sample of 45, 41 had their latest run started by the sync cron.

Change

src/revision.ts: resolveRevision() picks, in order of trust,

an explicit revision input (new, optional),
the pull request head on pull_request events (unchanged: that checkout is the merge commit, and the run should link to the branch head),
the commit checked out under package (git rev-parse HEAD, never throws),
GITHUB_SHA.

The chosen source is logged (Revision: <sha> (from checked-out HEAD).). Behaviour is identical for plain push events and pull requests, where the checkout and GITHUB_SHA agree. action.yml and the README document the new input. dist/ rebuilt with ncc.

Testing

__tests__/revision.test.ts: 8 cases for the precedence and the git lookup (real checkout, a non-git dir, a missing dir).
npm run all (build, format, lint, package, jest): clean; the existing main.test.ts still passes (9/9 total).
Not yet exercised on a real sync-triggered run. Suggested check after release: dispatch sync.yml on a fork whose upstream moved and confirm the Mayhem run's revision equals the rebased mayhem HEAD.

Rollout

Every rlenv fork pins ForAllSecure/mcode-action@v1, so once this ships in a v1.0.x release and the v1 tag moves, all forks pick it up on their next run without per-fork changes. Existing runs keep their old label until a new run lands.

🤖 Generated with Claude Code

The run's revision came from GITHUB_SHA, which is fixed when the workflow run starts. A job that checks out a different ref therefore labelled the run with a commit it did not build: a reusable workflow called with a freshly rebased commit, a workflow_run handler, an explicit `ref:` on actions/checkout. Downstream tooling that compares the run's revision with the branch head then reads a healthy run as stale. Seen on savantenvs/delta (2026-09-20): the weekly sync rebased `mayhem` to 6ac2067, called mayhem.yml with that ref, the fuzz job checked out and built commit-6ac2067, and the Mayhem run recorded 4b79bc9 — the pre-rebase head the cron started from. Revision now resolves, in order of trust: an explicit `revision` input; the pull request head on pull_request events (unchanged, the checkout there is the merge commit); the commit checked out under `package` (`git rev-parse HEAD`); GITHUB_SHA. The source is logged. A new optional `revision` input lets a workflow override it. Behaviour is unchanged for push events and pull requests, where the checkout and GITHUB_SHA agree. __tests__/revision.test.ts pins the precedence and the git lookup; dist rebuilt with ncc. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

github-actions · 2026-09-21T14:49:25Z

Mayhem Automated Code Testing Passed! ✔️ ✔️

✔️ 🎆 0 Defects Found

Testing details found at https://app.mayhem.security/forallsecure/mcode-action/lighttpd/172

…ITHUB_SHA Dispatch-only (it starts a real Mayhem run): check out the dispatched ref's parent into a second checkout, run the action against it, then read the run back from the Mayhem API and assert its revision is that parent — different from GITHUB_SHA. This is the reusable-workflow case where the fuzzed commit is not the one the workflow run started from. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

A workflow that exists only on a branch cannot be dispatched; main.yml can, and runs the branch's copy. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

TR1LON · 2026-09-21T17:50:36Z

Tried on a real run now. Added a dispatch-only CI job (revision-from-checkout in main.yml, guarded on workflow_dispatch because it starts a real Mayhem run). It checks out the dispatched ref's parent into a second checkout, runs the action against that, then reads the run back from the Mayhem API and asserts its revision:

Run 35633403754, branch revision-from-checkout, Mayhem run forallsecure/mcode-action/lighttpd/159:

GITHUB_SHA     : b2f6a2bddf334ac9a7b83f0737370f0d7329af7a
checked out    : 9348bfb72c72c619edcfadd52fbd9463d5b6aea3
Revision: 9348bfb72c72c619edcfadd52fbd9463d5b6aea3 (from checked-out HEAD).
recorded       : 9348bfb72c72c619edcfadd52fbd9463d5b6aea3
OK: the run records the commit that was actually checked out

Before this change the same job would have recorded b2f6a2b. This is the exact shape of the fleet's sync.yml → mayhem.yml call (the fuzzed commit is not the one the workflow run started from). The regular push/PR jobs in the same run also pass, so nothing changed for the normal case where checkout and GITHUB_SHA agree.

Not yet exercised on a fleet fork; the first weekly sync after the v1 bump will be the real confirmation (delta's next sync run should record the rebased mayhem HEAD).

Bump after merge: v1 currently points at main HEAD (8dd6c9e, moved 2026-09-04; last numbered tag is v1.0.9 from 2024, package.json already says 1.0.10). So after merging: tag v1.0.10 on the merge commit and move v1 to it. Every fork pins @v1, so that is the whole rollout. Happy to do it, or leave it to whoever normally cuts releases.

d-dot-one

If you're OK with the 1 comment, I'm OK with this PR :)

d-dot-one · 2026-09-21T18:52:11Z

+export function checkedOutRevision(cwd: string): string | undefined {
+  try {
+    const out = execFileSync("git", ["rev-parse", "HEAD"], {
+      cwd,
+      stdio: ["ignore", "pipe", "ignore"],
+    })
+      .toString()
+      .trim();
+    return /^[0-9a-f]{40}$/.test(out) ? out : undefined;
+  } catch {
+    return undefined;


This func silently swallows all git errors. Having the catch {return undefined} combined with stdio: [ignore, pipe, ignore] is basically equivalent to "no checkout present" and falls through to GITHUB_SHA with a normal log entry. The problem this PR is trying to solve is most likely to hit this

Agreed, a silent fallback there hides exactly the case this PR is for. Changed:

checkedOutRevision now returns {sha} or {error}, with git's own stderr in the error (stdio captures it instead of discarding it). Still never throws.

getConfig() emits a workflow warning whenever the revision ends up as GITHUB_SHA because the checkout could not be read, naming the cause and pointing at the revision input as the override. So a job that builds a different commit but has no readable checkout is flagged in the run summary, not just in a log line.

Explicit revision input and the pull-request head path are unchanged (no warning: those are deliberate choices, not fallbacks).

Tests updated to assert the error text for a non-git directory and a missing path (the test update landed one commit after the source change; both are on the branch now), npm run all clean, 9/9.

… read Review (d-dot-one): checkedOutRevision swallowed every git error and returned undefined, so a failed lookup fell through to GITHUB_SHA with an ordinary log line — the exact wrong-label case this change exists to fix, hidden. It now returns {sha} or {error} with git's own stderr, and getConfig() emits a workflow warning naming the cause whenever the revision ends up coming from GITHUB_SHA because of it, pointing at the `revision` input as the override. Explicit input and pull-request head are unaffected. Tests cover the error text for a non-git directory and a missing path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

sauliusbc

This adds quite a lot of changes, which I cannot verify wouldn't break every existing use of this action if still is shipped with @v1.

I would rather recommend just doing a simple additive behavioral change where:

revision input is added (like in this PR)
revision is first value to be considered at line 105, i.e.

    revision: getInput("revision")
      || seventPullRequest
        ? eventPullRequest.head.sha
        : process.env["GITHUB_SHA"] || "unknown",

Savantenvs migrated en-masse to pass-in revision input to mcode-action

sauliusbc

Actually, this probably looks like the easiest fix for the existing fleet of integrations as otherwise we'd have to rewrite 100s of integrations to pass-in --revision where appropriate.

ethan42 requested review from d-dot-one and sauliusbc September 21, 2026 17:37

TR1LON and others added 2 commits September 21, 2026 17:38

ci: run the revision end-to-end check from main.yml (dispatch only)

b2f6a2b

A workflow that exists only on a branch cannot be dispatched; main.yml can, and runs the branch's copy. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

d-dot-one approved these changes Sep 21, 2026

View reviewed changes

TR1LON and others added 2 commits September 21, 2026 19:19

test: checkedOutRevision reports git's error instead of undefined

83ef9d3

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

sauliusbc requested changes Sep 23, 2026

View reviewed changes

sauliusbc approved these changes Sep 23, 2026

View reviewed changes

TR1LON merged commit 3fbde05 into main Sep 23, 2026
20 of 23 checks passed

Navigation Menu

Record the commit the job actually checked out, not GITHUB_SHA - #255

Record the commit the job actually checked out, not GITHUB_SHA#255
TR1LON merged 5 commits into
mainfrom
revision-from-checkout

TR1LON commented Sep 21, 2026

Uh oh!

github-actions Bot commented Sep 21, 2026 •

edited

Loading

Uh oh!

TR1LON commented Sep 21, 2026

Uh oh!

d-dot-one left a comment

Uh oh!

d-dot-one Sep 21, 2026

Uh oh!

TR1LON Sep 21, 2026 •

edited

Loading

Uh oh!

sauliusbc left a comment

Uh oh!

sauliusbc left a comment •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

TR1LON commented Sep 21, 2026

Problem

Change

Testing

Rollout

Uh oh!

github-actions Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Mayhem Automated Code Testing Passed! ✔️ ✔️

✔️ 🎆 0 Defects Found

Uh oh!

TR1LON commented Sep 21, 2026

Uh oh!

d-dot-one left a comment

Choose a reason for hiding this comment

Uh oh!

d-dot-one Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

TR1LON Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sauliusbc left a comment

Choose a reason for hiding this comment

Uh oh!

sauliusbc left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Sep 21, 2026 •

edited

Loading

TR1LON Sep 21, 2026 •

edited

Loading

sauliusbc left a comment •

edited

Loading