Skip to content

Add reference results for the externally-hosted cases #934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MakisH merged 1 commit into from
Sep 23, 2026
+7 −7
Merged

Add reference results for the externally-hosted cases #934

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/generate-reference-results.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ jobs:
else
git add -A -- '**/reference-results/*.tar.gz'
fi
git commit -m "${{inputs.commit_msg}}"
git commit -m "${{inputs.commit_msg}}" || echo "::warning:: Could not commit the reference results - Ignore if only meant to be stored locally."
git push
- name: Archive system test logs
if: ${{ always() }}
Expand Down
2 changes: 1 addition & 1 deletion tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ To re-run one test locally:
2. `cd` into the test folder.
3. Run `./rerun-system-test.sh` (or `sh rerun-system-test.sh`).

The first time you run tests locally, you need to [initialize the mirror of third-party repositories](#running-locally). There are also some restricted cases, which you cannot run locally and are not included in the `release` test suite (see `restricted` test suite).
The first time you run tests locally, you need to [initialize the mirror of third-party repositories](#running-locally). There are also some restricted cases, which you cannot run locally and are not included in the `release` test suite (see `restricted` test suite). Similarly, externally-hosted have their reference results stored in the hosted runner; If you don't have these reference results, set `skip_compare: true` in the tests under the `external:` test suite.

The script rebuilds images, runs the tutorial, and (if present) runs fieldcompare with `--exit-code-from field-compare`, matching the Python runner. Compose paths are relative to the test folder (`..` is the parent `runs/` directory), so you can move the extracted tree elsewhere on a Linux host with Docker.

Expand Down
2 changes: 1 addition & 1 deletion tests/reference_versions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ FENICSX_ADAPTER_REF: "v1.0.1"
FMI_RUNNER_REF: "v0.2.1"
JULIA_BINDINGS_REF: "v3.4.0"
MICRO_MANAGER_REF: "bf5ea7b" # develop, July 7, 2026
OPENFOAM_ADAPTER_REF: "2c3062c" # develop, May 27, 2026
OPENFOAM_ADAPTER_REF: "v1.4.0"
PRECICE_REF: "v3.4.1"
PYTHON_BINDINGS_REF: "v3.4.0"
RUST_BINDINGS_REF: "v3.4.0"
Expand Down
8 changes: 4 additions & 4 deletions tests/tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -981,8 +981,8 @@ test_suites:
- left-openfoam
- right-openfoam
timeout: 480
skip_compare: true
reference_result: ./partitioned-heat-conduction-robin/reference_results/left-openfoam_right-openfoam.tar.gz
skip_compare: false # set to true if you don't have access to the reference results
reference_result: /home/precice/reference-results/partitioned-heat-conduction-robin/left-openfoam_right-openfoam.tar.gz

partitioned-heat-conduction-mixedbc:
external:
Expand All @@ -996,8 +996,8 @@ test_suites:
- left-openfoam
- right-openfoam
timeout: 480
skip_compare: true
reference_result: ./partitioned-heat-conduction-mixedbc/reference_results/left-openfoam_right-openfoam.tar.gz
skip_compare: false # set to true if you don't have access to the reference results
reference_result: /home/precice/reference-results/partitioned-heat-conduction-mixedbc/left-openfoam_right-openfoam.tar.gz

#####################################################################
## Restricted test cases
Expand Down
Loading