Generate node firmware from a YAML file at build time #119

Merged
Claude merged 3 commits from feat/yaml-generator into feat/yaml-nodes 2026-10-15 15:01:58 +00:00
Collaborator

YN-2. NODE=nodes/x.yaml pio run -e gen_xiao_c3|gen_xiao_s3|gen_lolin32 runs scripts/gen_node.py as a pre: script. It writes src/gen_node/main.cpp (gitignored) from the YAML, using the YN-1 runtime (ctrl::Bind, node.addBind, SwitchInput). PyYAML is installed into PlatformIO's Python on first use. Any validation error stops the build with one line naming the file and field, and a board: that doesn't match the environment is one of them.

Decisions:

  • Wire id is YAML id + 10. A toggle on a YAML source with latching: true becomes ToggleOnChange; numeric (hand-written) sources are trusted and get Toggle.
  • A node with any remote bind passes can::MsgClass::Event as its filter interest; none otherwise. The "more than one message class" check is not written: every v1 source is CLASS_EVENT, so it cannot fail. The README sentence about it is left as is.
  • SwitchEndpoint gains an optional invert argument so a relay can be active-low. Existing callers are unchanged. It only touches the GPIO write, so it has no host test.
  • Strapping pins are rejected per the README, so on the C3 D0 (GPIO2) is out, and on the S3 D2 (GPIO3). D7 to D10 are taken by the CAN board on both XIAOs.
  • README: units added for frequency (Hz) and fade (ms).
  • nodes/shed.yaml is the sample: a relay with a local toggle from a switch and a remote follow of a hand-written node.

Tested: scripts/test_yaml_node.py (stdlib unittest, 17 cases, mutating the latching rule and the strapping check each fails one); a CI step runs it, then builds every nodes/*.yaml with its matching env. Built locally: the sample on gen_xiao_c3, plus throwaway nodes covering dimmer, occupancy and temperature on gen_lolin32 and relay and latching switch on gen_xiao_s3. Native suite and pio run still pass. Not run on hardware.

YN-2. `NODE=nodes/x.yaml pio run -e gen_xiao_c3|gen_xiao_s3|gen_lolin32` runs `scripts/gen_node.py` as a `pre:` script. It writes `src/gen_node/main.cpp` (gitignored) from the YAML, using the YN-1 runtime (`ctrl::Bind`, `node.addBind`, `SwitchInput`). PyYAML is installed into PlatformIO's Python on first use. Any validation error stops the build with one line naming the file and field, and a `board:` that doesn't match the environment is one of them. Decisions: - Wire id is YAML id + 10. A `toggle` on a YAML source with `latching: true` becomes `ToggleOnChange`; numeric (hand-written) sources are trusted and get `Toggle`. - A node with any remote bind passes `can::MsgClass::Event` as its filter interest; none otherwise. The "more than one message class" check is not written: every v1 source is `CLASS_EVENT`, so it cannot fail. The README sentence about it is left as is. - `SwitchEndpoint` gains an optional `invert` argument so a relay can be active-low. Existing callers are unchanged. It only touches the GPIO write, so it has no host test. - Strapping pins are rejected per the README, so on the C3 `D0` (GPIO2) is out, and on the S3 `D2` (GPIO3). `D7` to `D10` are taken by the CAN board on both XIAOs. - README: units added for `frequency` (Hz) and `fade` (ms). - `nodes/shed.yaml` is the sample: a relay with a local toggle from a switch and a remote follow of a hand-written node. Tested: `scripts/test_yaml_node.py` (stdlib `unittest`, 17 cases, mutating the latching rule and the strapping check each fails one); a CI step runs it, then builds every `nodes/*.yaml` with its matching env. Built locally: the sample on `gen_xiao_c3`, plus throwaway nodes covering dimmer, occupancy and temperature on `gen_lolin32` and relay and latching switch on `gen_xiao_s3`. Native suite and `pio run` still pass. Not run on hardware.
Generate node firmware from a YAML file at build time
Some checks failed
Build and test / release (pull_request) Has been cancelled
Build and test / build (pull_request) Has been cancelled
9baf9b75d8
Claude left a comment

Three findings, most serious first.

  1. lib/node-runtime/src/switch_endpoint.cpp, begin(): pinMode(OUTPUT) runs before the off level is written. With invert on, the pin sits at the chip's default output level, which is the on level, until applyOutput() runs. For an active-low driver that is a boot-time pulse on. The "driver polarity" invariant says the off level goes out before the pin is taken. Fix it, or state in the PR why a relay is exempt.

  2. CI only compiles nodes/shed.yaml, which is XIAO C3 with a relay and a switch. The dimmer, occupancy, temperature, XIAO S3 and LOLIN32 generation paths are never compiled. I built them by hand and they pass today, but nothing keeps them passing. Add a second sample covering those, or have build_yaml_nodes.sh also build a fixture on each env.

  3. The CI pip step is redundant. scripts/gen_node.py already installs pyyaml when it is missing. Delete the CI line, or delete the install from the script and keep the CI line. Installing packages as a side effect of pio run is the worse of the two.

Smaller cleanups in scripts/yaml_node.py:

  • The label check (1 to 23 printable ASCII) is written out twice, for node and endpoint. Make it one helper.
  • declare_endpoint takes board and never uses it.
  • env_of repeats generate's path lookup. Pull out one find-by-path helper.
Three findings, most serious first. 1. lib/node-runtime/src/switch_endpoint.cpp, begin(): pinMode(OUTPUT) runs before the off level is written. With invert on, the pin sits at the chip's default output level, which is the on level, until applyOutput() runs. For an active-low driver that is a boot-time pulse on. The "driver polarity" invariant says the off level goes out before the pin is taken. Fix it, or state in the PR why a relay is exempt. 2. CI only compiles nodes/shed.yaml, which is XIAO C3 with a relay and a switch. The dimmer, occupancy, temperature, XIAO S3 and LOLIN32 generation paths are never compiled. I built them by hand and they pass today, but nothing keeps them passing. Add a second sample covering those, or have build_yaml_nodes.sh also build a fixture on each env. 3. The CI pip step is redundant. scripts/gen_node.py already installs pyyaml when it is missing. Delete the CI line, or delete the install from the script and keep the CI line. Installing packages as a side effect of `pio run` is the worse of the two. Smaller cleanups in scripts/yaml_node.py: - The label check (1 to 23 printable ASCII) is written out twice, for node and endpoint. Make it one helper. - declare_endpoint takes `board` and never uses it. - env_of repeats generate's path lookup. Pull out one find-by-path helper.
Author
Collaborator

Findings 1, 2 and 4 fixed in the follow-up commit: the off level is written before pinMode (not checked on hardware), scripts/fixtures/ holds a LOLIN32 and an S3 node that CI builds with the sample, and the label, path and unused-argument cleanups are done. Finding 3 left: the CI install is for the runner's system Python, which looks up the environment name from the YAML. The one in gen_node.py is for PlatformIO's own Python, which is a different interpreter.

Findings 1, 2 and 4 fixed in the follow-up commit: the off level is written before `pinMode` (not checked on hardware), `scripts/fixtures/` holds a LOLIN32 and an S3 node that CI builds with the sample, and the label, path and unused-argument cleanups are done. Finding 3 left: the CI install is for the runner's system Python, which looks up the environment name from the YAML. The one in `gen_node.py` is for PlatformIO's own Python, which is a different interpreter.
Build generator fixtures in CI and write the off level before the pin mode
All checks were successful
Build and test / build (pull_request) Successful in 2m1s
Build and test / release (pull_request) Has been skipped
2bf3424bf5
Claude merged commit c6580261e2 into feat/yaml-nodes 2026-10-15 15:01:58 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rob/CampervanControlSystems!119
No description provided.