Conversation
McCode ships components, library snippets and example instruments that are
maintained in other people's repositories. ChopperLib.cmake fetched one such
contribution, mcstas-chopper-lib, by hiding a FetchContent call in a bespoke
CMake module -- which works, but puts the record of where those files come
from in the one place nobody looking for a component would think to open.
Replace it with a generic mechanism. Each contribution declares itself in a
small JSON manifest named "<something>.ext", placed in the directory its
files are populated into, so a developer browsing mcstas-comps/contrib/
finds either the .comp itself or an .ext file naming the upstream repository,
the release it is pinned to, and the SHA256 of every file taken from it.
cmake/Modules/External.cmake reads those manifests. One
mccode_install_externals() call per directory mirrors the install( DIRECTORY
... ) rule already there, and each manifest's files land at its own position
relative to that directory -- which is how the one-directory-per-instrument
layout under examples/ falls out without special-casing it.
A manifest may name files individually by URL or lift them out of a release
tar.gz, mixing the two if it likes; archives are unpacked once and shared
across every manifest naming the same one. The sha256 of each file is
mandatory: a pinned tag states an intention, and only the hash makes a build
fail loudly when the bytes behind that tag change -- which matters especially
for GitHub's auto-generated source archives, regenerated on demand and not
byte-stable across GitHub's own compression changes.
Downloads are content-addressed in MCCODE_EXTERNALS_CACHE. Packagers who may
not reach the network can pre-populate that cache or point
MCCODE_EXTERNALS_LOCAL at the files, and set MCCODE_EXTERNALS_OFFLINE=ON so
a gap in what they provided fails the configure rather than quietly
downloading.
buildscripts/mcext computes the hashes so they need not be transcribed by
hand ("update", which also repoints a manifest at a new upstream tag) and
re-checks them against upstream ("check", non-zero exit on any mismatch --
the one to wire into CI).
chopper-lib stays pinned at v4.1.0 and installs exactly where it did before.
Manifests themselves are not installed; the install( DIRECTORY ... ) rules
covering those directories now carry PATTERN "*.ext" EXCLUDE.
Note that this needs CMake >= 3.19 for string( JSON ); the floor bump is a
separate commit.
The *.ext manifest machinery added in the previous commit parses JSON with string( JSON ), which CMake gained in 3.19 (and unpacks release archives with file( ARCHIVE_EXTRACT ), 3.18). Declare what we actually need. This is a floor raise only, kept separate so it can be reverted on its own if some downstream consumer turns out to be stuck below 3.19. Reverting it does not break an external-contribution build on a 3.19+ toolchain; it only makes the declared minimum wrong again. The cmake_policy( VERSION 3.17.0 ) pins under tools/ are deliberately left alone: raising the floor should not quietly change policy behaviour in the directories that explicitly fixed it. Bumping those is a separate decision.
2 tasks
Contributor
|
@g5t it looks good but current bandwidth is all reserved for https://github.com/mccode-dev/PaNRAID/blob/main/README.md next week... I will try to have a look at this detail late in / after next week... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Record external contributions where they land
ChopperLib.cmakefetchedmcstas-chopper-libfor inclusion of external contributed library and components in the McStas distribution. The use of a bespoke CMake module meant that the record of where those files come from was separate from where a user might look for their definitions.Each contribution now declares itself in a JSON manifest named
<name>.ext, placed in the directory its files are populated into:mcstas-comps/share/chopper-lib.extchopper-lib.c,chopper-lib.hmcstas-comps/contrib/chopper-lib.ext.compfiles…/examples/Tests_optics/<Instr>/<Instr>.extcmake/Modules/External.cmakereads them: onemccode_install_externals()call per directory, mirroring theinstall( DIRECTORY ... )rule already there. Files come from individual URLs or a releasetar.gz(unpacked once, shared across manifests naming the same one), and everysha256is mandatory.buildscripts/mcextcomputes the hashes (updatea manifest at a new tag) and re-checks them against upstream (check, non-zero exit on mismatch for CI use). Format and options:docs/EXTERNAL-CONTRIBUTIONS.md.Two commits, deliberately split
*.extmanifests — the mechanism.string( JSON ). Keptseparate so it can be reverted alone if something downstream is stuck below 3.19;
reverting only makes the declared minimum wrong, it doesn't break the build on a
3.19+ toolchain. The
cmake_policy( VERSION 3.17.0 )pins undertools/areleft alone on purpose.
Notes
PATTERN "*.ext" EXCLUDE).MCCODE_EXTERNALS_CACHEor pointMCCODE_EXTERNALS_LOCALat the files, then setMCCODE_EXTERNALS_OFFLINE=ONso agap fails the configure instead of quietly downloading.
FetchContent_Populate(chopperlib) is deprecatedwarning.Declaration of use of AI-tools
Claude Code used Opus 5 to implement this external contributed fetch refactor.
Development OS / boundary conditions
Please describe what OS you developed and tested your additions on, and if any special dependencies are required:
Developed on Linux.
PR Checklist for contributing to McStas/McXtrace
For a coherent and useful contribution to McStas/McXtrace, please fill in relevant parts of the checklist:
My contribution contains something else