fix: build release binaries from the source tree, not the checkout - #1317
Merged
Merged
Conversation
Four crates in the lock run `git rev-parse HEAD` in their build script and bake the answer into what they compile. The canonical builder mounts the repository at /workspace with CARGO_HOME inside it, so they resolved this repository rather than their own and recorded the commit being released. Release bytes then changed with every commit even when no compiled input did, which is what stalled v0.33.0: the Base Registry image advisory baseline records reviewed layer digests in one commit and compares them against the candidate built from the next one, so the whole-image fingerprint assertion could never converge again and no renewal of it could succeed. Run the builder container with GIT_CEILING_DIRECTORIES=/workspace so repository discovery stops at the mount point. All four crates already fall back to their packaged-build value when git fails, and no workspace crate reads git, so nothing else changes. The inner invocation now refuses a payload built without the ceiling, making it part of the canonical container contract instead of a caller setting. Also read the staged payload back and fail the build when a binary still contains the exact source commit, as a regression check for a path the ceiling does not cover. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This was referenced Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
Release area. A release binary must be a function of the source tree, not of
the checkout it was built in. Four crates in
Cargo.lockrungit rev-parse HEADin their build script and bake the answer into what theycompile:
wasmtime-internal-cache48.0.2COMPILER_VERSIONCARGO_PKG_VERSIONcranelift-codegen0.135.2VERSION=0.135.2-<rev>0.135.2typst-utils0.15.1TYPST_COMMIT_SHAwasm-bindgen-shared0.2.xWBG_VERSIONoption_env!)Each is meant to read its own development checkout. The canonical builder
mounts this repository at
/workspaceand putsCARGO_HOMEinside it, sorepository discovery from a vendored build script found
/workspace/.gitandrecorded the commit being released. Release bytes therefore changed on every
commit even when no compiled input changed.
This runs the builder container with
GIT_CEILING_DIRECTORIES=/workspace,which stops repository discovery at the mount point.
Why this is blocking v0.33.0
The Base Registry image advisory baseline records reviewed layer digests in
one commit and compares them, via
whole_image_fingerprint_equals, against acandidate built from a later commit. Candidate run 35747918154 (source
125f67b37) failed exactly there:Layers 0-20 and 22 were byte-identical to the reviewed base; only the BReg
application layer moved, and inside it only
/usr/local/bin/breg. The twooccurrences in that binary were located at offsets 1472394
(
(engine ..)-125f67b37b424..., wasmtime, full 40) and 2195806(
0.135.2-125f67b37, cranelift-codegen, 9 chars). No compiled input changedbetween the reviewed revision
6a460c2cfand125f67b37.The assertion could never converge again, and no renewal of the baseline
could succeed, because every renewal commit changes the bytes it is
renewing against. This makes the gate convergent again.
Nothing else in the release path sees this. The weekly repeatability proof
rebuilds one published tag, so the commit is identical on both sides of its
comparison and the defect is invisible to it.
Blast radius
GIT_CEILING_DIRECTORIESis a git-internal discovery limit, not a build flag.All four crates degrade to their packaged-build value rather than failing, so
no build breaks. Verified there is no other git reader:
build.rsreads git.registry-platform-buildinfo/build.rsembeds only
DISPLAY_VERSIONandIS_RELEASE_BUILD, both from environment.vergen,built,git2,gix, orshadow-rsinCargo.lock.the full-40 or 9-character form (
breg: 1 full, 2 abbreviated; casework,discovery, evidence, relay, scheduling: zero of both).
typst-utilsandwasm-bindgen-sharedare in the lock but not in anyrelease asset; they are listed for completeness.
Verified empirically on git 2.54.0 that a ceiling equal to the repository root
blocks
git rev-parse HEADfrom a subdirectory (exit 128, "not a gitrepository") while still resolving with the cwd at the root itself. Cargo runs
build scripts below
/workspaceand runs itself at/workspace, so this isthe property required: build scripts are blocked, Cargo's own git access is
not.
Adjacent addition, easy to drop
check_source_commit_absentreads every staged binary back before the payloadis checksummed and fails the build if one still contains the exact source
commit. It is a regression check for a path the ceiling does not cover, for
example a build script that names the repository directory itself rather than
discovering it. It matches the full commit only, so a build script embedding
an abbreviation alone passes here and is caught by the ceiling instead. The
commit is supplied as
RELEASE_SOURCE_COMMIT, taken fromRELEASE_SOURCE_SHAon the release path and from the local checkout otherwise; when neither
resolves, the check says so on stderr and passes rather than failing silently.
If this is out of scope for a release-unblocking change, say so and I will
drop the helper, its call site, and its four tests. The ceiling alone fixes
the defect.
Also in the canonical container contract
The inner
RELEASE_BUILDER_READYguard now requiresGIT_CEILING_DIRECTORIES=/workspacealongside the existingrepo_root,CARGO_HOME,CARGO_TARGET_DIR, and tag checks. The ceiling is part of therecipe, not a setting a caller can drop.
Evidence
Written test-first; the four new tests failed with
check_source_commit_absent: command not foundbefore the helper existed.python3 -m unittest release/scripts/test_zig_glibc_compiler.pypython3 -m unittest release/scripts/test_registry_release.pypython3 -m unittest release/scripts/test_release_candidate.pypython3 -m unittest release/scripts/test_release_repeatability_workflow.pypython3 -m unittest release/scripts/test_check_gates_inventory.pypython3 -m unittest release/scripts/test_release_workflow_structure.pypython3 -m unittest release/scripts/test_release_rehearsal.pypython3 -m unittest release/scripts/test_check_release_source_model.pybash -n release/scripts/build-release-binaries.shshellcheck release/scripts/build-release-binaries.shSC1091info on sourcing../glibc-floor.envshfmt -dis inconclusive: it wants to reformat pre-existing case statementsand
[[ ]]continuations throughout the file. Neither shellcheck nor shfmtruns in this repo's CI and the file's existing style already diverges from
shfmt defaults, so formatting was left alone and the additions follow the
file's 2-space convention.
release_candidate.pyonly regex-greps this recipe forcp ... dist/image-bin/<name>lines, so the additions do not affect how thepayload inventory is derived. Both
release-candidate.ymlandrelease-rehearsal.ymlalready setRELEASE_SOURCE_SHA, so the release pathalways knows its exact commit, and both cache keys hash
build-release-binaries.sh, so this change correctly invalidates the buildcache.
The convergence proof is the next candidate itself: after this lands, BReg's
bytes depend only on source content, and the pending advisory-baseline renewal
touches only
release/security/*.jsonandrelease/scripts/test_check_advisory_baselines.py, neither of which reaches abinary.
Notes
Release provenance and security baseline change; needs explicit review.
It changes what goes into published release bytes.
COMPILER_VERSIONinwasmtime-internal-cachekeys an on-disk module cache, so the fallback makesthat key a function of the wasmtime version rather than of an unrelated
repository's commit, which is the correct behaviour for a shipped binary.
VERSIONincranelift-codegenis diagnostic only. Neither is a securitycontrol.
No published bytes are replaced and no tag moves. The candidate that exposed
this failed before its reverify-and-attest step, so nothing was sealed,
attested, tagged, or published; v0.33.0 will be cut from a new candidate.
Follow-up in this release, not in this PR: renew the BReg advisory baseline
against the post-fix image per
release/OPERATIONS.md, then request a freshcandidate.
DCO
Signed-off-bytrailer.