Skip to content

ci: install poppler through aptneed, so a mirror outage stops blocking releases - #92

Merged
tannevaled merged 2 commits into
mainfrom
fix/poppler-through-aptneed
Nov 13, 2026
Merged

tannevaled merged 2 commits into
mainfrom
fix/poppler-through-aptneed

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

On 2026-11-11 this repository could not be released, and nothing about this repository was wrong.

The step

sudo apt-get update && sudo apt-get install -y --no-install-recommends poppler-utils

failed four times in half an hour on one merge commit, every time with Temporary failure resolving 'azure.archive.ubuntu.com', while the pull request it came from had been green on this same workflow minutes earlier. ghrelease will not tag while main is red — correctly — so a third party's DNS held every release hostage, and the step had no way to say which of the two it was.

What changes

GOBIN=/usr/local/bin go install github.com/go-fleettools/fleettools/cmd/aptneed@latest
sudo aptneed --need pdftoppm poppler-utils

aptneed rewrites /etc/apt/apt-mirrors.txt to the canonical archive after a mirror failure. That one file is what every source on a GitHub runner resolves through, so naming archive.ubuntu.com in a source would change nothing — the source still reads the mirrorlist.

It also refuses to retry a package apt cannot locate, because that is a name and not weather — except where apt could not refresh its lists either, in which case the two are indistinguishable from the text and it says so rather than guessing.

--need pdftoppm, because apt saying yes is not the binary being there

A package renamed upstream installs cleanly and leaves the job without its judge, which is how a suite comes to skip the very thing it was measuring and go green doing it. The two render tests already t.Fatal when CI is set and pdftoppm is missing (place_test.go:410, poster_render_test.go:114); --need asks the same question one step earlier, where it can still be answered. Binaries already on PATH are a silent pass, so a warm runner pays nothing.

Ordering, which is the part that nearly bit me

aptneed is pinned @latest, which resolves to a tag. It merged to main this afternoon and was in no tag, so pushing this branch then would have failed the go install and reddened this lane for a reason that is not this repository's — the same class of confusion the change exists to remove.

It went out only after go-fleettools/fleettools v0.111.0 was released and go list -m ...@latest actually resolved to it; the tag existing is not the same thing, because the module proxy serves @latest from its own cache. Measured both ways before pushing: @latest returned v0.110.0 immediately after the tag, and v0.111.0 once the proxy had it.

aptneed is go-fleettools/fleettools#252, merged after five rounds of review.

actionlint clean.

Closes #90.

🤖 Generated with Claude Code

tannevaled and others added 2 commits November 11, 2026 16:09
…g releases

⛔⛔ A THIRD PARTY'S DNS HELD EVERY RELEASE HOSTAGE. On 2026-11-11 this step

    sudo apt-get update && sudo apt-get install -y --no-install-recommends poppler-utils

failed FOUR times in half an hour on one merge commit, every time with
`Temporary failure resolving 'azure.archive.ubuntu.com'`, while the pull
request it came from had been green on this same workflow minutes earlier —
and then started failing pull requests too. ghrelease will not tag while main
is red, correctly, so nothing about this repository was wrong, nothing could be
released, and the step had no way to say which of the two it was (#90).

aptneed (go-fleettools/fleettools) rewrites /etc/apt/apt-mirrors.txt to the
canonical archive after the first mirror failure — that one file is what every
source on a runner resolves through, so naming archive.ubuntu.com in a SOURCE
would change nothing — and it refuses to retry a package apt cannot locate,
which is a name and not weather.

⛔ --need pdftoppm, because apt saying yes is not the binary being there. The
two render tests already t.Fatal when CI is set and pdftoppm is missing
(place_test.go:410, poster_render_test.go:114), which is the gate that keeps
the foreign judge from being silently skipped; --need is the same question
asked one step earlier, where it can still be answered.

actionlint clean.

Closes #90.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
⛔ THE FIRST PUSH OF THIS BRANCH WENT RED, AND NOT FOR THE REASON I HAD
GUARDED AGAINST. I held it back until go-fleettools/fleettools v0.111.0 was
released AND `go list -m …@latest` resolved to it, because `@latest` is a tag
and the module proxy serves it from its own cache. That part worked: the
runner logged `go: downloading …/fleettools v0.111.0`.

What failed is one line further down:

    go: …/cmd/aptneed@latest: …/fleettools@v0.111.0 requires go >= 1.27.4
        (running go 1.27.3; GOTOOLCHAIN=local)

fleettools v0.111.0 declares `go >= 1.27.4` and this workflow pinned 1.27.3.

⭐ AND GOTOOLCHAIN=local IS WHY THIS IS A CLEAN FAILURE RATHER THAN A SILENT
SWAP. Without it Go would have downloaded 1.27.4 on its own and the job would
have measured a toolchain nobody chose — the thing this repository sets that
variable to prevent. The refusal names both versions and the remedy is one
line, which is what a pinned toolchain is for.

go.mod still says `go 1.27.2` and is left alone: that is the language version
this module needs, not the toolchain the runner must have, and nothing here
requires it to move.

actionlint clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled

Copy link
Copy Markdown
Contributor Author

Correcting one thing in the second commit message before it is squashed in.

It says "fleettools v0.111.0 declares go >= 1.27.4", which is true but reads as though the requirement arrived with that release — mine. It did not. Checked across the last three tags:

v0.109.0/go.mod:3:go 1.27.4
v0.110.0/go.mod:3:go 1.27.4
v0.111.0/go.mod:3:go 1.27.4

So this workflow's 1.27.3 pin would have refused any recent fleettools command, not just aptneed. The release did not break it; the pin had already drifted below what the fleet's tools require, and this branch is simply the first thing to ask for one.

Worth knowing for other repositories in the fleet that pin a toolchain and install fleettools commands in CI: the same pin blocks them the moment they try, and the failure names both versions, so it is at least a legible one.

Nothing to change in the diff — 1.27.4 is correct either way.

@tannevaled
tannevaled merged commit 72a16a1 into main Nov 13, 2026
1 check passed
@tannevaled
tannevaled deleted the fix/poppler-through-aptneed branch November 13, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: a mirror outage installing poppler blocks every release, and nothing can tell it from a real absence

1 participant