Document approaches for site builds on top of EESSI by casparvl · Pull Request #778 · EESSI/docs

casparvl · 2026-06-04T11:18:09Z

No description provided.

… the mkdocs.yml

ocaisa

Found some typos

first attempt at writing the shared fs section

…, after a reboot, the stratum 0 overlay won't mount

…cript. Also, added a full, copyable script combining all the previous components.

Neves-P

Looks really nice! This is my first pass of comments, as of know I only focused on the text. This week I will follow the documentation in practice and review again.

Neves-P · 2026-06-23T08:49:10Z

+
+Another thing to consider is to create a secondary user with `aws iam create-user --user-name <name>` and attach a very limited policy to it (e.g. only read/write/list on buckets, nothing else). Then, create credentials for this user with `aws iam create-access-key --user-name <name>` and provide those credentials to the EESSI build bot and Stratum 0 machines. That way, if that token is compromized, the impact is minimized (e.g. the token can at least not be used to create new IAM idententies, etc).
+
+## Setting up the EESSI build bot


I'd consider separating this into a new page. The build bot is somewhat self contained and the page is quite long already.

casparvl · 2026-07-22T14:34:04Z

+
+Another thing to consider is to create a secondary user with `aws iam create-user --user-name <name>` and attach a very limited policy to it (e.g. only read/write/list on buckets, nothing else). Then, create credentials for this user with `aws iam create-access-key --user-name <name>` and provide those credentials to the EESSI build bot and Stratum 0 machines. That way, if that token is compromised, the impact is minimized (e.g. the token can at least not be used to create new IAM idententies, etc).
+
+## Setting up the EESSI build bot


I realize these docs lack one crucial thing: setting up a site_config_script that sets EESSI_SITE_INSTALL_FORCE and EESSI_SITE_SOFTWARE_PATH_PREFIX

I.e. at SURF, our site_config_script looks like this:

# To build on top of EESSI, we need to software.eessi.io repository to be mounted next to our own repository # The bot/build.sh script does this when the EESSI_SITE_INSTALL_FORCE environment variable is set # Other build scripts will also respect this variable where needed in order to make sure that 'building on top' # of EESSI is possible export EESSI_SITE_INSTALL_FORCE=1 echo "Value of EESSI_SITE_INSTALL_FORCE: $EESSI_SITE_INSTALL_FORCE" # We also need to set a prefix that our installations should end up in # The build scripts should take this prefix, and construct the final EESSI_SITE_SOFTWARE_PATH out of it # that the EESSI-extend module expects export EESSI_SITE_SOFTWARE_PATH_PREFIX=/cvmfs/software.surf.nl/versions/2025.06/ echo "Value of EESSI_SITE_SOFTWARE_PATH_PREFIX: $EESSI_SITE_SOFTWARE_PATH_PREFIX"

One thing to figure out might be how to avoid hard-coding the version in here :)

Scratch that, this should not be needed anymore... I think we do this differently now... We automatically set these here https://github.com/EESSI/software-layer-scripts/blob/ca929cd7ef32a9fcd79fafd4e0d5c362a1fff452/bot/build.sh#L156 . That happens if the repo_name in the repos config file is unequal to software.eessi.io or dev.eess.io essentially.

I think this does warrant a small explanation of why EESSI_SITE_SOFTWARE_PREFIX, EESSI_SITE_INSTALL and friends are not needed with this workflow. Maybe we don't even need an explanation, but just a mention.

I'm saying this because going through the whole docs - especially the shared file system installs - you get the impression that at least doing EESSI_SITE_INSTALL is necessary for site installations (which it is, but that happens behind the scenes).

Neves-P

Just two comments, suggestions. I would have liked to have been more thorough, but overall I did two passes to the text and managed to get to the point of installing the bot (had to wait a bit for the internal S3 and maintenance).

In my mind, after addressing these comments we can either wait a little longer and I spin up a new test CVMFS infrastructure to also test that part, or merge before that and address any thing else we spot in a later PR.

Overall I think the instructions are very nice! I'm not sure I am able to put myself in the shoes of a complete novice to EESSI, but they seem very appropriate for that target audience.

Neves-P · 2026-08-04T10:16:59Z

+
+Another thing to consider is to create a secondary user with `aws iam create-user --user-name <name>` and attach a very limited policy to it (e.g. only read/write/list on buckets, nothing else). Then, create credentials for this user with `aws iam create-access-key --user-name <name>` and provide those credentials to the EESSI build bot and Stratum 0 machines. That way, if that token is compromised, the impact is minimized (e.g. the token can at least not be used to create new IAM idententies, etc).
+
+## Setting up the EESSI build bot


I think this does warrant a small explanation of why EESSI_SITE_SOFTWARE_PREFIX, EESSI_SITE_INSTALL and friends are not needed with this workflow. Maybe we don't even need an explanation, but just a mention.

I'm saying this because going through the whole docs - especially the shared file system installs - you get the impression that at least doing EESSI_SITE_INSTALL is necessary for site installations (which it is, but that happens behind the scenes).

Neves-P · 2026-08-04T15:38:05Z

+``` { .bash .copy }
+curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
+unzip awscliv2.zip
+sudo ./aws/install


This can be installed in userland, and ideally I'd do so. That's how I normally install it in the dev.eessi.io bots and it works well.

Neves-P

Another small detail that I spotted now and am adding here so I don't forget.

Neves-P · 2026-09-01T14:39:17Z

The last commit message was supposed to read: "Account for custom prefix installs for .cvmfsdirtab"

After that is reviewed, we need to account for the changes in the tarball ingestion step too. That is blocked until EESSI/filesystem-layer#278 is merged.

Co-authored-by: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com>

Co-authored-by: Bob Dröge <b.e.droge@rug.nl>

Neves-P

Suggestions with updates accounting for VERSIONS_SUBPATH workflow in ingest-tarball.sh.

Co-authored-by: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com>

bedroge · 2026-09-21T13:51:04Z

+# We will leverage a script from eessi-bot-software-layer (for signature verification - optional)
+git clone https://github.com/EESSI/eessi-bot-software-layer.git
+
+# We will leverage a script from filesystem-layer (for tarball ingestion)
+git clone https://github.com/EESSI/filesystem-layer.git


Having these here will break things for a second run, as these dirs will already exist. We should either add a check, or maybe just let people do this manually and add a variable for the paths to these dirs at the top? I think I'd prefer the latter, makes it a bit more flexible in case you want to (temporarily) modify these scripts, e.g. for testing.

bedroge · 2026-09-21T13:52:29Z

+    if [ $? -eq 0 ]; then
+        echo "Tarball signature file downloaded."
+    else
+        echo "WARNING: Failed to download tarball signature file. Continuing to next tarball (not ingesting ${filename})."
+        # No point in continuing this loop iteration, we'll fail the signature verification check anyway
+        continue
+    fi


This basically does mean that it's not optional, as it will now entirely skip tarballs with a signature file. Maybe we should have another variable at the top that allows people to enable/disable the signature checking? If it's enabled but the sig file(s) cannot be downloaded -> hard error for that tarball. If it's disabled, don't even try.

Add initial files for each site-building approach

a4d5de8

casparvl changed the title ~~Add initial files for each site-building approach~~ Document approaches for site builds on top of EESSI Jun 4, 2026

Caspar van Leeuwen and others added 5 commits June 4, 2026 17:12

Made a start with the site builds as a CVMFS repo

259e935

Started to write Stratum 0 install docs. Work in progress

6aebcec

Slight rephrasing of site-build overview section and add new pages to…

dcf695f

… the mkdocs.yml

Merge branch 'main' into site_builds

87ccf16

Corrected typo

c887a09

ocaisa requested changes Jun 9, 2026

View reviewed changes

Caspar van Leeuwen added 2 commits June 9, 2026 15:20

More or less completed the docs on how to setup a Stratum 0

729cf88

Finished description for Stratum 0 and Stratum 1

c31746d

casparvl commented Jun 10, 2026

View reviewed changes

Comment thread docs/site_build/site_cvmfs.md

Caspar van Leeuwen and others added 9 commits June 11, 2026 10:09

Finished a few more chapters

e4fac7e

Small update

b9205ca

Started section on installing EESSI build bot on a machine

7b027fc

Further work on deploying bot on machine

3f49e7b

first attempt at writing the shared fs section

8fe6e19

Merge pull request #5 from bedroge/site_builds_shared_fs

0cddb6b

first attempt at writing the shared fs section

Make sure we add instructions to stop _and disable_ autofs. Otherwise…

ed6f41a

…, after a reboot, the stratum 0 overlay won't mount

Merge branch 'site_builds' of github.com:casparvl/docs into site_builds

5aa289b

Add some debugging instructions

d200d12

Neves-P mentioned this pull request Jun 30, 2026

Add documentation on how to build on top of EESSI as a site - and include the use of EESSI_SITE_SOFTWARE_PREFIX in that #768

Open

Caspar van Leeuwen added 2 commits June 30, 2026 16:23

Added an image, rewrote some parts, fixed some formatting

92f27e0

Fixed some typos

6ea3bef

casparvl marked this pull request as ready for review June 30, 2026 14:36

Caspar van Leeuwen and others added 2 commits July 1, 2026 17:36

Added logic to do signature verification to the automatic ingestion s…

acfcb9e

…cript. Also, added a full, copyable script combining all the previous components.

Fixed codespell issues

2bba8e8

Neves-P linked an issue Jul 7, 2026 that may be closed by this pull request

Add documentation on how to build on top of EESSI as a site - and include the use of EESSI_SITE_SOFTWARE_PREFIX in that #768

Open

Neves-P self-requested a review July 7, 2026 08:16

Neves-P requested changes Jul 20, 2026

View reviewed changes

Fix typo

e0724fa

Neves-P added 5 commits July 20, 2026 16:04

Fix missing word

7bef7b8

Fix markdown

51a0d0a

Add line break

0cc3ec7

Bash highlight

bf3f6f8

Use monospaced font for path

e18965a

casparvl commented Jul 22, 2026

View reviewed changes

Neves-P requested changes Aug 4, 2026

View reviewed changes

Neves-P requested changes Aug 7, 2026

View reviewed changes

Comment thread docs/site_build/site_cvmfs.md Outdated

Neves-P mentioned this pull request Aug 28, 2026

Clean up dev.eessi.io-specific code (assume it's doing a site installation) and improve tarball creation step for site installations EESSI/software-layer-scripts#295

Merged

Neves-P added 2 commits September 1, 2026 15:28

Fix link

daeb7b0

Fix link

2bc09af

bedroge reviewed Sep 2, 2026

View reviewed changes

Comment thread docs/site_build/site_cvmfs.md Outdated

bedroge and others added 2 commits September 2, 2026 09:53

Apply batched suggestions from code review

3547ee9

Co-authored-by: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com>

Apply batched suggestions from code review

2296339

Co-authored-by: Bob Dröge <b.e.droge@rug.nl>

bedroge reviewed Sep 15, 2026

View reviewed changes

Comment thread docs/site_build/site_cvmfs.md Outdated

Update docs/site_build/site_cvmfs.md

579898c

Co-authored-by: Bob Dröge <b.e.droge@rug.nl>

Neves-P requested changes Sep 18, 2026

View reviewed changes

Neves-P requested a review from bedroge September 18, 2026 13:02

Apply batched suggestions from code review

cbde340

Co-authored-by: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com>

bedroge reviewed Sep 21, 2026

View reviewed changes


		Another thing to consider is to create a secondary user with `aws iam create-user --user-name <name>` and attach a very limited policy to it (e.g. only read/write/list on buckets, nothing else). Then, create credentials for this user with `aws iam create-access-key --user-name <name>` and provide those credentials to the EESSI build bot and Stratum 0 machines. That way, if that token is compromized, the impact is minimized (e.g. the token can at least not be used to create new IAM idententies, etc).

		## Setting up the EESSI build bot

Conversation

casparvl commented Jun 4, 2026

Uh oh!

ocaisa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Neves-P left a comment

Choose a reason for hiding this comment

Uh oh!

Neves-P Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

casparvl Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

casparvl Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

casparvl Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

Neves-P Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Neves-P left a comment

Choose a reason for hiding this comment

Uh oh!

Neves-P Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Neves-P Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Neves-P left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Neves-P commented Sep 1, 2026

Uh oh!

Uh oh!

Uh oh!

Neves-P left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bedroge Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

bedroge Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

casparvl Jul 22, 2026 •

edited

Loading

Neves-P Aug 4, 2026 •

edited

Loading

Neves-P Aug 4, 2026 •

edited

Loading