Skip to content

feat(state_machine): generate the example's HFSM from its model at build time - #776

Merged
finger563 merged 9 commits into
mainfrom
feature/generate-hfsm-in-build
Sep 5, 2026
Merged

feat(state_machine): generate the example's HFSM from its model at build time#776
finger563 merged 9 commits into
mainfrom
feature/generate-hfsm-in-build

Conversation

@finger563

Copy link
Copy Markdown
Contributor

The example shipped ~2400 lines of generated C++ and no model. The machine couldn't be regenerated, edited, or checked against anything — and it had drifted: regenerating from the model with the current generator produces roughly two thousand different lines (after clang-format), three years of generator fixes the example never received.

So the model is the source now, and the C++ is a build product like an object file.

  • main/Complex.json is checked in; the three generated files are not.
  • CMake runs the generator at configure time and lists the model in CMAKE_CONFIGURE_DEPENDS, so editing the model regenerates on the next build, not the next clean.
  • It generates with --no-support, so only the machine is produced. The shared runtime — state_base.hpp, the history states, magic_enum.hpp — comes from this component. A second copy beside the generated machine would land on the include path in front of espp's, and espp's are the ones the rest of the codebase is built against.

What it needs

node >= 18, and nothing else — npx fetches the generator on demand. Two escape hatches:

# work against a local generator checkout; also how to build offline
idf.py -DHFSM_GEN_COMMAND="node;/path/to/webgme-hfsm/bin/hfsm-gen.js" build

The esp-idf CI image has no node (IDF's own tools.json doesn't ship it), so that matrix entry installs it first. Worth being explicit: this is a new build-time dependency for this one example, which was the deliberate trade for not versioning generated code.

Editing the machine

The model opens in the HFSM Playground — browser-based, no install, no server — or straight to this example's machine. Save back over main/Complex.json and rebuild.

Verified

Actually built, not just configured, on esp-idf v6.1 / esp32:

  • clean set-target + build from an empty tree, with freshly generated code three years newer than what was committed
  • the fresh code compiles against espp's component headers — checked separately with only the three model files on the include path, so the generator's own headers couldn't satisfy it by accident
  • edited Complex.json, rebuilt, confirmed the change reached the compiled source without a clean

Blocking on two things

  1. --no-support is not released yet — it's Add --no-support, for projects that vendor the runtime already finger563/webgme-hfsm#241. The pin here is that PR's commit.
  2. Once feat(motorgo): Add APIs for more customized use (single motor, delayed motor init, etc.) #241 and the v1.8.0 release land, HFSM_GEN_SPEC should become webgme-hfsm@^1.8.0. It's a git ref today because npx webgme-hfsm@1.7.0 doesn't work at all — installing that package and running the CLI fails outright, which is the packaging bug fixed in Make the WebGME server optional so the CLI can be installed finger563/webgme-hfsm#238 and not yet released.

A note on the vendored runtime

I checked whether this component's state_base.hpp / history headers had gone stale against the generator's, since the generated code had. They have not. All 19 shared method bodies are byte-identical once comments and whitespace are normalised; the differences are espp's own improvements (declaration/definition split into src/state_base.cpp, explicit constructors, doxygen). And magic_enum runs the other way: espp vendors 0.9.5, the generator ships 0.8.0. Nothing to update here — if anything the generator is the one behind.

finger563 and others added 2 commits September 4, 2026 23:26
…ild time

The example shipped ~2400 lines of generated C++ and no model. The
machine could not be regenerated, edited, or checked, and the code had
drifted: regenerating Complex.json with the current generator produces
roughly two thousand different lines, three years of generator fixes
that the example never got.

The model is the source now, and the C++ is a build product like an
object file. main/Complex.json is checked in; the three generated
files are not. CMake runs the generator at configure time and lists
Complex.json in CMAKE_CONFIGURE_DEPENDS, so editing the model
regenerates on the next build rather than the next clean.

Only the model-specific files are taken from the generator's output.
It also emits its own state_base.hpp, deep_history_state.hpp,
shallow_history_state.hpp and magic_enum.hpp, and this component
already provides those -- espp's are the adapted ones the rest of the
codebase expects, and putting the generator's copies on the include
path would shadow them.

Needs node >= 18; npx fetches the generator on demand, pinned to a
commit so builds are reproducible. -DHFSM_GEN_COMMAND points the build
at a local checkout instead, which is also how to build offline. The
esp-idf CI image has no node, so that matrix entry installs it.

Verified on esp-idf v6.1 / esp32: clean configure and build, and an
edit to Complex.json reaching the compiled source on the next build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E4hRq1q5VGPjNn2dnkkqBy
The build was taking the generator's whole output and copying three
files out of it. The generator can now be asked for those three
directly, so the copy step goes away.

It matters beyond tidiness: the four files being skipped are a second
copy of the runtime this component already provides, and they would
sit on the include path in front of espp's. espp's are the ones the
rest of the codebase is built against, and its magic_enum (0.9.5) is
newer than the generator's (0.8.0).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E4hRq1q5VGPjNn2dnkkqBy
Copilot AI lite review requested due to automatic review settings September 5, 2026 04:36
The pin was a git ref for two reasons, and the release settles both:
`npx webgme-hfsm@1.7.0` could not run at all, and --no-support did not
exist. 1.8.0 is the first release with either.

^1.8.0 rather than an exact version: reproducible enough for a build
that regenerates from a checked-in model, and picks up generator
fixes without a commit here.

Verified against the published package: a clean esp-idf v6.1 / esp32
build, and the npm CLI's output is byte-identical to the repo's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E4hRq1q5VGPjNn2dnkkqBy
@finger563

Copy link
Copy Markdown
Contributor Author

Updated now that v1.8.0 is published — both blockers are gone.

HFSM_GEN_SPEC is webgme-hfsm@^1.8.0 from npm instead of a git commit. It was a git ref only because npx webgme-hfsm@1.7.0 could not run at all (the packaging bug) and --no-support did not exist; 1.8.0 is the first release with either.

Verified against the published package rather than assuming: a clean set-target + build on esp-idf v6.1 / esp32, and the npm CLI's output is byte-identical to a local checkout's.

^1.8.0 rather than an exact pin — reproducible enough for a build that regenerates from a checked-in model, and it picks up generator fixes without a commit here. Say the word if you'd rather it were exact.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Moves the Complex HFSM example to treat the JSON model as the source of truth and generate the C++ state machine during the build, removing committed generated artifacts that had drifted from the generator output.

Changes:

  • Add Complex.json model to source control and stop committing generated C++/metadata for the example.
  • Generate the HFSM at CMake configure time via npx (with --no-support) and compile the generated .cpp from the build directory.
  • Update documentation and CI to reflect/install the new Node-based build-time dependency.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
components/state_machine/example/main/Complex_metadata.json Removes committed generator metadata file.
components/state_machine/example/main/Complex_generated_states.hpp Removes committed generated header (now build output).
components/state_machine/example/main/Complex_generated_states.cpp Removes committed generated source (now build output).
components/state_machine/example/main/Complex_event_data.hpp Removes committed generated event data header (now build output).
components/state_machine/example/main/Complex.json Adds the HFSM model as the checked-in source of truth.
components/state_machine/example/main/CMakeLists.txt Adds configure-time generation of the HFSM C++ via npx and compiles generated output.
components/state_machine/example/README.md Documents generation workflow, Node requirement, and offline/local-generator option.
components/state_machine/README.md Adds general guidance on generating state machines with webgme-hfsm / Playground.
.github/workflows/build.yml Installs Node/NPM for this example build in CI and runs idf.py build.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/build.yml Outdated
Comment thread components/state_machine/README.md
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

✅Static analysis result - no issues found! ✅

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The configured npm version is unavailable, and obsolete generated files can survive incremental regeneration.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 9/9 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread components/state_machine/example/main/CMakeLists.txt Outdated
Comment thread components/state_machine/README.md Outdated
finger563 and others added 2 commits September 5, 2026 09:01
…tive npx, stale-artifact cleanup

- build.yml: the state_machine example's node guard only checked presence, so a
  base image / apt source with node <18 would still fail. Check node's major
  version and install Node 20 (>=18) from NodeSource when it is missing or too
  old.
- README: pin the documented generator to `npx -y -p webgme-hfsm@^1.8.0` --
  `-y` for non-interactive/CI use, and the version pin (now that 1.8.0 is
  released) matches the example's CMake spec for reproducible generation.
- example CMake: the generator only overwrites the files it emits, so a machine
  renamed/removed in the model left an obsolete generated .cpp in the persistent
  binary dir for CMake to keep compiling. Wipe + recreate the output dir before
  generation so a regen cannot succeed against stale artifacts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The example carried ~50 lines of CMake to find npx, run the generator
and wire up the output. Anyone generating a second machine would have
copied it, and the copies would have drifted.

espp_generate_hfsm() now lives in the component's
project_include.cmake, which ESP-IDF includes before any component's
CMakeLists, so every component can call it. The example is four lines.

It finds the generated file names rather than making the caller name
them: they come from the machine's name in the model, so hardcoding
Complex_generated_states.cpp meant knowing what your own model was
called and keeping that in step by hand.

The call needs an `if(NOT CMAKE_BUILD_EARLY_EXPANSION)` guard, which
is not obvious and is documented at both call site and README:
ESP-IDF's early requirements pass includes component CMakeLists
WITHOUT project_include.cmake, so the function does not exist yet and
the build fails with "Unknown CMake command". That pass does not read
SRCS either, so the empty variables are what it expects.

The cache variables are ESPP_-prefixed now that they are the
component's interface rather than one example's local detail.

Verified on esp-idf v6.1 / esp32: clean build, a model edit reaching
the compiled source without a clean, the documented
ESPP_HFSM_GEN_COMMAND override, and both error paths (a model that is
not there, a misspelled argument) reporting which call was wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E4hRq1q5VGPjNn2dnkkqBy
@finger563

Copy link
Copy Markdown
Contributor Author

Done — espp_generate_hfsm() now lives in the component's project_include.cmake, which ESP-IDF includes before any component's CMakeLists, so any component can call it. The example went from ~50 lines of CMake to four:

if(NOT CMAKE_BUILD_EARLY_EXPANSION)
  espp_generate_hfsm(
    MODEL "${CMAKE_CURRENT_LIST_DIR}/Complex.json"
    OUTPUT_DIR "${CMAKE_CURRENT_BINARY_DIR}/hfsm"
    SOURCES_VAR hfsm_srcs
    INCLUDE_DIR_VAR hfsm_inc)
endif()

idf_component_register(SRCS "hfsm_example.cpp" ${hfsm_srcs}
                       INCLUDE_DIRS "." ${hfsm_inc})

It also finds the generated file names rather than making the caller name them — they come from the machine's name in the model, so the old hardcoded Complex_generated_states.cpp meant knowing what your own model was called and keeping that in step by hand. WITH_SUPPORT and NAMESPACE are there for callers who need them.

One wart worth knowing about, since it isn't obvious and I hit it: the call needs the if(NOT CMAKE_BUILD_EARLY_EXPANSION) guard. ESP-IDF's early requirements pass includes component CMakeLists without project_include.cmake, so without the guard the build fails with Unknown CMake command "espp_generate_hfsm". That pass doesn't read SRCS either, so the empty variables are exactly what it expects. It's documented at the call site and in the component README so the next person doesn't have to rediscover it.

The cache variables are ESPP_-prefixed now that they're the component's interface rather than one example's local detail: ESPP_HFSM_GEN_SPEC, ESPP_HFSM_GEN_COMMAND.

Verified on esp-idf v6.1 / esp32 — clean build, a model edit reaching the compiled source without a clean, the documented ESPP_HFSM_GEN_COMMAND override actually working against a local checkout, and both error paths (missing model, misspelled argument) naming the call that was wrong. Your stale-artifact cleanup and the --yes npx behaviour are carried into the function.

@finger563
finger563 requested a balanced review from Copilot September 5, 2026 14:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Comment on lines +104 to +109
# Clear stale artifacts first. The generator only overwrites what it
# emits, so a machine renamed or removed in the model would leave an
# obsolete .cpp behind in the (persistent) binary dir for CMake to go
# on compiling.
file(REMOVE_RECURSE "${HFSM_OUTPUT_DIR}")
file(MAKE_DIRECTORY "${HFSM_OUTPUT_DIR}")
Comment thread .github/workflows/build.yml
Comment thread components/state_machine/project_include.cmake
Comment thread components/state_machine/project_include.cmake Outdated
Comment thread components/state_machine/project_include.cmake
Co-authored-by: finger563 <213467+finger563@users.noreply.github.com>
@finger563

Copy link
Copy Markdown
Contributor Author

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply the suggestion exactly.

Do not make changes beyond what is described in the linked review thread.

Co-authored-by: finger563 <213467+finger563@users.noreply.github.com>

Copilot AI commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply the sug...

Addressed the review thread in commit 9aa95ec: OUTPUT_DIR is now required to resolve inside CMAKE_BINARY_DIR before recursive deletion, generated sources remain sorted, and CI installs Node 20 via an explicit signed NodeSource apt repository setup instead of piping a remote script to bash. The implementation and README already use the released webgme-hfsm@^1.8.0 package.

@finger563

Copy link
Copy Markdown
Contributor Author

@copilot your latest changes broke the state machine ci build. Fix them.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The CI command has broken nested shell quoting, and the documented consumer setup omits a required component dependency.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

components/state_machine/project_include.cmake:37

  • ^1.8.0 is not a reproducible pin: npx may select any later 1.x generator, changing the generated C++ (or breaking the build) without a repository change. Since generated files are no longer checked in or locked, pin the exact generator version and update it explicitly.
# Pinned to a minor range: reproducible enough for a build that
# regenerates from a checked-in model, and picks up generator fixes
# without a commit here. 1.8.0 is the first release the CLI can be
# installed from at all, and the first with --no-support.
set(ESPP_HFSM_GEN_SPEC "webgme-hfsm@^1.8.0"

components/state_machine/README.md:26

  • The caret range can resolve to later 1.x releases, so the statement that this makes generation reproducible is incorrect. Use the same exact generator version as CMake (for example, webgme-hfsm@1.8.0) and update the adjacent explanation accordingly.
npx -y -p webgme-hfsm@^1.8.0 hfsm-gen my_machine.json -o generated

(-y skips npx's install prompt for non-interactive/CI use; the @^1.8.0
pin matches the example's CMake so generation is reproducible.)


- **Files reviewed:** 10/10 changed files
- **Comments generated:** 3
- **Review effort level:** Balanced
</details>

Comment thread .github/workflows/build.yml
Comment on lines +42 to +43
idf_component_register(SRCS "main.cpp" ${hfsm_srcs}
INCLUDE_DIRS "." ${hfsm_inc})
Comment thread components/state_machine/project_include.cmake
Co-authored-by: finger563 <213467+finger563@users.noreply.github.com>

Copilot AI commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

@copilot your latest changes broke the state machine ci build. Fix them.

Fixed in 910adeb. The matrix command now avoids inner single quotes, so esp-idf-ci-action can embed it in its single-quoted shell command.

@finger563
finger563 merged commit ad69a5f into main Sep 5, 2026
156 checks passed
@finger563
finger563 deleted the feature/generate-hfsm-in-build branch September 5, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants