Skip to content

test: make changelog contract robust to rollover; bump ubuntu restart budget - #309

Merged
StefanSteiner merged 2 commits into
tableau:mainfrom
StefanSteiner:fix/contract-test-and-restart-timing
Sep 7, 2026
Merged

test: make changelog contract robust to rollover; bump ubuntu restart budget#309
StefanSteiner merged 2 commits into
tableau:mainfrom
StefanSteiner:fix/contract-test-and-restart-timing

Conversation

@StefanSteiner

Copy link
Copy Markdown
Contributor

Two main-side test fixes that currently make main's full test suite red. Both are test-only (hyperdb-mcp/tests/); no product code, versions, release-please files, or workflows are touched.

Fix A — smoke_demo_and_changelog_contract regression from #307

smoke_demo_and_changelog_contract asserted the release's KV/export/routing claims only against the ## [Unreleased] slice of the embedded mcp CHANGELOG. PR #307 (the rc.3 changelog rollover) moved every bullet out of ## [Unreleased] into ## [1.0.0-rc.3], emptying the Unreleased slice and failing the token/heading/hyper-format export assertions.

This is a real regression from #307, surfaced only later: #307 was docs-only, so CI's paths-ignore: **/*.md skipped the Rust suite for it, and the break didn't appear until a code PR ran the tests.

Approach — assert against the current release window (union of ## [Unreleased] + the most-recent dated section), not the whole changelog. A new current_release_window helper concatenates the ## [Unreleased] slice with the single newest dated ## [x.y.z] section. Before a release the claims live under Unreleased; after a rollover they live under the newest dated section; the window spans both, so a future docs: rollover can't silently rebreak the contract. I deliberately did not search the whole changelog: tokens like doctor / health port could plausibly appear in older historical entries (## [0.5.0], ## [0.1.1]), which would let a deleted claim still pass and weaken the check. Bounding to the newest dated section preserves the "these claims are documented for the current release candidate" intent while excluding stale history. Every existing assertion's intent is preserved; no check was weakened or removed.

Red-before-green (cargo test -p hyperdb-mcp --test readme_tests):

  • Before: exit 101 — 12 failures (must contain at least one bullet under ### added/fixed/changed, does not account for "doctor"/"resolved_database"/"resource_busy"/"health port"/"engine_busy"/"bar_orientation"/"label_values"/"show_legend"/"y_scale").
  • After: test result: ok. 9 passed; 0 failed.

Fix B — ubuntu restart tests time out under CI load

engine_recovers_after_hyperd_killed and hyperd_monitor_detects_killed_hyperd_and_restarts time out on ubuntu-latest under CI load: the liveness monitor polls on a 5 s tick (daemon::run::HYPERD_POLL_INTERVAL), so a kill costs up to ~5 s just to be noticed, then recovery must drop the dead process, cold-spawn a fresh hyperd, rebind, and republish STATUS. The old 12 s budget (~5 s detection + ~7 s slack) is too tight when a shared runner's cold spawn eats several seconds. They pass 4/4 locally and the third restart test passes on the same CI run, so this is a false timeout, and it predates the #304 IPC work.

This is a budget bump, not a coverage change. The three magic 12s are replaced with a documented named constant RESTART_READINESS_BUDGET_SECS = 45 (~40 s headroom past the detection tick, still bounding a genuinely wedged daemon). The macOS-only #[cfg_attr(target_os = "macos", ignore)] is unchanged — it is not extended to Linux, so the Linux crash-recovery coverage added by #279 (and depended on by #286's fix) stays intact on Linux/Windows.

Cross-references issue #305 (the standing daemon-test timing decision for the macOS-ignored set); this timing fragility is now evidence the budget question spans Linux too. Related flakes: #300, #288.

Local --ignored run (macOS host, these are macOS-ignored):

cargo test -p hyperdb-mcp --test daemon_tests -- --ignored --exact \
  engine_recovers_after_hyperd_killed \
  hyperd_monitor_detects_killed_hyperd_and_restarts \
  client_report_triggers_restart_after_kill
test result: ok. 3 passed; 0 failed; ... finished in 16.17s

16 s locally sits well under 45 s; the extra headroom absorbs a loaded runner's slower detection + cold spawn without ever masking a real hang.

Gate (all green, real output)

  • cargo fmt --all -- --check → exit 0
  • cargo clippy --workspace --all-targets --all-features -- -D warnings → exit 0, 0 warnings
  • cargo test -p hyperdb-mcp663 passed; 0 failed; 17 ignored (the previously-failing contract test now passes)
  • RUSTDOCFLAGS="-D warnings" cargo doc --no-deps → exit 0, 0 warnings
  • npx markdownlint-cli2 → 0 issues in 68 files

Notes

Fix A: smoke_demo_and_changelog_contract asserted KV/export/routing claims
only against the `## [Unreleased]` slice of the embedded mcp CHANGELOG.
PR tableau#307 (the rc.3 changelog rollover) moved every bullet into
`## [1.0.0-rc.3]`, emptying Unreleased and failing the test. Because tableau#307
was docs-only, CI's `paths-ignore: **/*.md` skipped the Rust suite, so the
break surfaced only on a later code PR. Assert instead against the current
release window (`## [Unreleased]` + the most-recent dated section) via a new
`current_release_window` helper, so a future rollover can't silently
rebreak it. The window is bounded to the newest dated section so a stale
token in an ancient entry can't satisfy a deleted-claim check.

Fix B: engine_recovers_after_hyperd_killed and
hyperd_monitor_detects_killed_hyperd_and_restarts time out on
ubuntu-latest under CI load — a 5s monitor tick plus cold hyperd spawn
against a 12s readiness budget. Replace the three magic `12`s with a named
RESTART_READINESS_BUDGET_SECS = 45 and document the derivation. Budget bump
only; the macOS-only ignore is unchanged, so Linux/Windows crash-recovery
coverage from tableau#279/tableau#286 stays. See tableau#305.
The constant's only use sites are the three `#[cfg(unix)]` restart tests
and their Unix-only helpers, so on Windows it compiled but was never
referenced and `clippy -D warnings` (windows-latest) rejected it as dead
code. Gate the definition with `#[cfg(unix)]` so it exists exactly where
it's used — the honest fix, not `#[allow(dead_code)]`.
@StefanSteiner
StefanSteiner merged commit bea4855 into tableau:main Sep 7, 2026
17 checks passed
StefanSteiner added a commit to StefanSteiner/hyper-api-rust that referenced this pull request Sep 7, 2026
…query

The two Linux-only daemon restart failures in tableau#310 are a test-harness bug,
not a regression in the restart path. `find_hyperd_pid_for_endpoint` selected
the socket owner with

    lsof -nP -t -c hyperd <socket path>

`lsof` ORs its list-selection options unless `-a` is given, so that query
returns every process whose command begins with `hyperd` *in addition to* the
socket's owner, and `-t` prints PIDs lowest-first. Whenever more than one
hyperd is alive -- routine in CI, where each test starts its own daemon while
the previous test's engine is still shutting down -- the helper returns the
oldest hyperd rather than this endpoint's, and the test kills a process it
does not own. Its own hyperd stays up, so phase 1 of
`wait_for_live_hyperd_after_kill` spins until the budget expires: a
deterministic full-budget timeout, which is why raising 12s to 45s in tableau#309
did not help.

Over TCP the equivalent selection (`-iTCP:<port> -sTCP:LISTEN`) names exactly
one listener, which is why these tests were green before the transport moved
to a socket path shared across restarts.

Identify the engine from hyperd's own lock file instead. hyperd writes
`<socket path>.pid` (bare decimal PID, O_EXCL) before it binds and unlinks it
on clean exit, so it names the one process serving this endpoint. A leftover
file can name a dead or recycled PID, so confirm the PID still is a hyperd
before returning it.

Also make the readiness helper report which phase expired and what it last
observed. A bare "didn't restart in time" is what kept this undiagnosed
across two CI runs.

Refs tableau#310
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant