Fix macOS nightly smoke failures for pdf-standard and Julia plot tests - #14938
Merged
Merged
Conversation
The nightly macOS leg is the first time the smoke suite runs on macOS, and it hit two gaps unrelated to the built binary. veraPDF was only installed on Linux, while the pdf-standard fixtures only exclude Windows. ua-missing-title is the one fixture that asserts on veraPDF's own output, so on macOS it rendered without validation and the expected WARN never appeared. The runner ships Java 21, so veraPDF installs fine there. The Julia crossref figure tests plot with Plots/GR. GR.jl does not recognize a julia-engine (QuartoNotebookRunner) worker as an embedded display, so on macOS GKS launches its interactive gksqt window. On the headless runner gksqt does not reliably start, and the render either blocks until the 10 minute test timeout or the worker dies. This is a GR/QuartoNotebookRunner behavior to be reported upstream, so these four tests are skipped on macOS for now.
Collaborator
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since #14706, the nightly built-version workflow runs the smoke suite on macOS, the only place it runs there. Two tests have failed every night since:
docs/smoke-all/pdf-standard/ua-missing-title.qmdanddocs/crossrefs/julia-subfig.qmd. Neither is specific to the built binary, a dev-mode run on a macOS runner shows the same behavior.Root cause
The veraPDF install step in
test-smokes.ymlonly runs on Linux, while the pdf-standard fixtures only exclude Windows.ua-missing-title.qmdis the one fixture asserting on veraPDF's own verdict (PDF validation failed for ua-2), so on macOS it rendered, printed "verapdf is not installed" and the expected warning never came.The Julia crossref figure tests plot with Plots/GR through the julia engine. GR.jl only picks a file-based workstation when it detects IJulia, Pluto, VS Code or Atom. A QuartoNotebookRunner worker matches none of them, so on macOS GKS launches its interactive
gksqtwindow. On the headless runnergksqtdoes not reliably start: the render either blocks until the 10 minute test timeout (worker waiting on thegksqtchild) or the worker dies withTerminatedWorkerException. Which one happens is racy,julia-subfig.qmdjust lost the race every night. Linux has no display, so GKS never startsgksqtthere.Fix
veraPDF is now installed on all non-Windows runners. The macOS runner ships Java 21, which veraPDF needs.
The four Julia Plots tests in
smoke/crossref/figures.test.ts(julia.qmd,julianative.qmd,julia-subfig.qmd,julianative-subfig.qmd) are skipped on macOS. Thegksqtbehavior affects any headless macOS julia-engine render, not only our CI, so it will be reported upstream rather than worked around here. SettingGKSwstypein the CI environment was considered but not adopted for that reason.Test plan
ua-missing-title.qmdpassesgksqtchild process, GR thread blocked on it) and the worker crash on repeated Julia Plots renderssmoke/crossref/figures.test.tson Windows: all 8 tests still run and pass