diff --git a/Biofirm_Generative_Model.md b/Biofirm_Generative_Model.md index e41a03e..85aaf16 100644 --- a/Biofirm_Generative_Model.md +++ b/Biofirm_Generative_Model.md @@ -22,15 +22,23 @@ symmetric `observation_bins` count (an odd number >= 3); the default of 3 preserves the below/at/above observation model, while larger odd counts (e.g. 5) discretize the state around the target more finely. +Variable and relationship mappings are canonicalized by name at construction, +including dependency accumulation and random-draw assignment. Hash-equivalent +configurations replay in that order; preserve original artifacts and runtime +provenance for earlier insertion-order-dependent runs. + ## Active inference The active-inference controller discretizes each variable into observation states around the target using the variable's configured `observation_bins`. It constructs the PyMDP `A`, `B`, `C`, and `D` matrices with JAX-compatible arrays, uses an empirical prior, performs state inference followed by policy inference, -and records the selected action. The action space is the three-step -decrease/hold/increase contract. Deterministic action selection is the default. -Stochastic selection requires a configured seed and uses a JAX PRNG key. +and records the effective control action. The action space remains the three-step +decrease/hold/increase contract. Action IDs are `0` for control below `-1e-12`, +`1` within `[-1e-12, 1e-12]`, and `2` above `1e-12`. Custom and generated decisions +must satisfy the same finite, bounded control/action rule before a transition. +Deterministic action selection is the default. Stochastic selection requires a +configured seed and uses a JAX PRNG key. The generative-model structure is specified in Generalized Notation Notation at [`models/biofirm_ecosystem_variable.gnn.md`](models/biofirm_ecosystem_variable.gnn.md); @@ -66,3 +74,8 @@ configuration snapshot and SHA-256 hash, controller name, package version, seed, summary, and immutable transition records. Consumers must validate the complete document before analysis or visualization; missing, extra, non-finite, or tampered fields are errors. +Validation also checks initial values and zero initial controls, bounds noise by +three configured standard deviations, reconstructs clipped next-state values and +recomputes posterior entropy from stored beliefs. Rejected historical documents +remain evidence and must not be rewritten to pass these checks; keys and schema +version remain unchanged. diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 1c0d4f2..056c45c 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -9,6 +9,12 @@ When no `--config` is supplied, the packaged default is used non-finite numbers, invalid ranges, duplicate or alias-colliding variable names, dependency cycles, and incomplete action vectors. +Variable, relationship-target and dependency mappings are frozen in canonical +name order at construction. Hash-equivalent configurations therefore assign +random draws and accumulate relationship effects in the same order. This changes +replay semantics for earlier insertion-order-dependent runs: retain their original +artifacts and package-version provenance rather than regenerating evidence. + ## Top-level keys | Key | Type | Required | Default | Meaning | @@ -69,6 +75,8 @@ acyclic. The active-inference controller builds the PyMDP `A`, `B`, `C`, and `D` matrices with JAX-compatible arrays using each variable's `observation_bins`. The action space is the fixed three-step decrease/hold/increase contract. +Action IDs describe effective control: `0` below `-1e-12`, `1` within +`[-1e-12, 1e-12]`, and `2` above `1e-12`, including zero control strength. ## Output definition @@ -94,11 +102,7 @@ The action space is the fixed three-step decrease/hold/increase contract. "unit": "index" } }, - "relationships": { - "biodiversity": { - "dependencies": { "water_quality": 0.1 } - } - }, + "relationships": {}, "controller": { "name": "active-inference", "stochastic": false diff --git a/README.md b/README.md index f354b91..dc366ac 100644 --- a/README.md +++ b/README.md @@ -67,6 +67,11 @@ controller identity, and package version. Invalid configuration, unavailable optional dependencies, malformed provider responses, and failed controller calls produce errors; they do not create empty success artifacts. +`parse-description` keeps signed decimal and leading-decimal ranges within their +original sentences. Extracted values retain units and original evidence +offsets; nearest-variable attribution and near-equidistant ambiguity reporting +are unchanged. A range cannot borrow a variable alias from another sentence. + `scenarios` runs the same seeds across multiple validated ecosystem configurations and reports per-scenario summaries plus a final-uncertainty estimate, so scenario variants can be compared without asserting calibrated @@ -78,11 +83,15 @@ Live research writes a JSON/Markdown pair for each region/persona plus a manifest containing integrity hashes. Every artifact records the request hash, prompt version, source hash, a provider-cost estimate, a human review state, and a set of structured findings extracted verbatim from the provider -response (with hedge-aware confidence). Resume is allowed only when both files, -their hashes, the prompt version/hash, and the source hash agree. Structural -validation establishes artifact integrity; it never certifies the factual -quality of citations or research claims. Research execution is split into -independently rerunnable acquire/prepare/execute/parse/render stages. +response (with hedge-aware confidence). Each complete pair is hashed and +checkpointed to the manifest before another provider job starts. A later +provider or rendering failure leaves committed pairs resumable; an incomplete +pair is not a completed manifest entry. Resume requires both files, their +hashes, the prompt version/hash, and the source hash to agree. Manifest costs +estimate committed artifact entries, not billed provider usage. Structural +validation establishes artifact integrity, never factual quality of citations +or research claims. Execution remains split into independently rerunnable +acquire/prepare/execute/parse/render stages. ## Configuration diff --git a/TODO.md b/TODO.md index a84aed2..b6838ca 100644 --- a/TODO.md +++ b/TODO.md @@ -14,6 +14,25 @@ workflows, P2 documentation and repository coherence, P3 experimental scientific and product expansion. Every item lands with tests, documentation, and an explicit acceptance check before it is marked complete. +## 2026-11-13 scoped qualification + +The independent `tests/test_research.py` and `tests/test_text_business.py` +modules passed all 37 cases without loading the PyMDP backend. This qualifies +the BF-C per-pair research checkpoint and BF-D decimal-range repairs, not live +provider evidence or a native simulation gate. Their current consumer guidance +is updated. The BF-F single-variable configuration example no longer references +undeclared `water_quality`; the validator's rejection contract is unchanged. + +At that qualification boundary, the complete four-file acceptance was blocked: +the prepared Python 3.12 environment had PyMDP 1.0.4 rather than 1.0.5, so BF-A, +BF-B and BF-E remained unqualified. On 2026-11-16, the unchanged frozen lock was +successfully provisioned with PyMDP 1.0.5 in an isolated integration environment. +Those originally authored software repairs are now selected with their +regressions and behavior documentation for native acceptance. This records source +selection and a resolved runtime prerequisite, not a passing gate or scientific +acceptance. Historical measurements, owner decisions and scientific prerequisites +are unchanged. + ## Completed/Closed Implemented in source, tests, and CI. Preserved here for auditability; do not diff --git a/docs/configuration.md b/docs/configuration.md index 7f98530..ae1421d 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -18,6 +18,10 @@ environment. All commands validate the same schema before running, so `biofirm validate --config path.json` is the fastest way to check a file. +The reference's single-variable example uses `"relationships": {}`. A +dependency must name another declared variable; validation rejects undeclared +sources. + ## What validation rejects Unknown fields, non-finite numbers, invalid ranges, duplicate or @@ -46,6 +50,9 @@ never commit a key. - The canonical JSON serialization of the validated configuration is hashed (SHA-256) and snapshotted into every simulation artifact, so a run can be audited against the exact configuration that produced it. +- Variable, relationship-target and dependency maps are frozen in canonical name + order, so hash-equivalent configurations use the same random-draw and summation + order. Preserve older insertion-order-dependent artifacts and provenance. - `output.directory` (default `artifacts`) is the directory used when a command writes by default, and `output.write_by_default` controls whether `simulate`/`sweep` write without an explicit `--output`. @@ -58,6 +65,9 @@ or `business`. The active-inference controller builds PyMDP `A`, `B`, `C`, and `observation_bins`; the action space is the fixed three-step decrease/hold/increase contract. See [`../CONFIGURATION.md`](../CONFIGURATION.md) for the parameter table. +Action IDs describe effective signed control, including zero and near-zero +strengths, using the `1e-12` threshold in the reference. Custom decisions must +obey the same finite, bounded control/action contract before simulation succeeds. ## Related diff --git a/docs/gate.md b/docs/gate.md index 6801b53..c5b160c 100644 --- a/docs/gate.md +++ b/docs/gate.md @@ -25,16 +25,16 @@ document. | `sync` | 0.0s | 0.0s | Lock the environment for the selected interpreter. | | `deps` | 0.0s | 0.0s | Verify uv.lock matches the manifests. | | `lint` | 1.0s | 0.1s | ruff check and ruff format --check on src tests tools. | -| `types` | 18.0s | 0.3s | Strict mypy on src/biofirm. | -| `test` | 74.0s | 24.6s | pytest with the 90% coverage floor (the full suite). | -| `test-fast` | 27.0s | 18.4s | pytest without coverage and without tests/test_gnn.py. | -| `links` | 3.0s | 1.0s | tools/verify_repository.py: relative links, source data, notebooks, credential patterns, required documents. | +| `types` | 18.0s | 0.4s | Strict mypy on src/biofirm. | +| `test` | 74.0s | 29.4s | pytest with the 90% coverage floor (the full suite). | +| `test-fast` | 27.0s | 14.8s | pytest without coverage and without tests/test_gnn.py. | +| `links` | 3.0s | 0.3s | tools/verify_repository.py: relative links, source data, notebooks, credential patterns, required documents. | | `docs` | 1.0s | 0.4s | biofirm validate and biofirm validate-data. | -| `bandit` | 2.0s | 1.0s | bandit on src and tools. | -| `build` | 28.0s | 10.1s | Build sdist and wheel into dist/. | -| `install` | 32.0s | 22.5s | Install dist/ into a clean venv and exercise the CLI outside the checkout, from the locked requirements. | -| `secret` | 54.0s | 23.5s | gitleaks scan with a synthetic negative control. | -| `gnn` | 5.0s | 7.8s | Pinned GNN conformance: the shared gnn.importers reader from the pinned revision, then tests/test_gnn.py. | +| `bandit` | 2.0s | 0.4s | bandit on src and tools. | +| `build` | 28.0s | 3.3s | Build sdist and wheel into dist/. | +| `install` | 32.0s | 17.7s | Install dist/ into a clean venv and exercise the CLI outside the checkout, from the locked requirements. | +| `secret` | 54.0s | 9.3s | gitleaks scan with a synthetic negative control. | +| `gnn` | 5.0s | 0.9s | Pinned GNN conformance: the shared gnn.importers reader from the pinned revision, then tests/test_gnn.py. | `gate` and `all` are meta arms and are not timed here: `gate` checks this document and `all` runs the other arms in order. `test-fast` had no CI arm @@ -48,16 +48,16 @@ once per command. ## Measured facts -- Full suite (`test`): **115 passed** with - **90.58% coverage** against the +- Full suite (`test`): **160 passed** with + **90.91% coverage** against the 90% floor. -- Fast arm (`test-fast`): **102 passed**. +- Fast arm (`test-fast`): **145 passed**. It runs the same suite without coverage and without `tests/test_gnn.py`, which the `gnn` arm owns. - GNN conformance (`gnn`): **15 passed**. - The full suite reports 2 skips from - `tests/test_gnn.py`: its pinned-parser checks need the export the `gnn` arm - writes, so they skip there and run here. + The full suite reports 0 skips from + `tests/test_gnn.py`. Its pinned-parser checks run when a prepared export is + available, otherwise they skip; the separate `gnn` arm requires that export. The preserved pre-gate baseline reported **119 passed** at @@ -73,9 +73,9 @@ CI compares them. | --- | --- | --- | | Host | `Darwin/arm64` | `Darwin/arm64` | | Python | `3.12.16` | `3.12.16` | -| uv | `0.13.7` | `0.13.7` | -| `uv.lock` SHA-256 | not recorded | `ba33e645ea9521dbd6708dc830afabe681f57a6c7f4a711f3a6e956295049aaa` | -| Recorded | `2026-11-12` | `2026-11-12T09:55:02-0800` | +| uv | `0.13.7` | `0.13.8` | +| `uv.lock` SHA-256 | not recorded | `e6d9bd0a85a6ba11b554b2f73032dd0ba58ec37e11e9f7a0c36a6f03b7ca6fff` | +| Recorded | `2026-11-12` | `2026-11-16T13:26:10-0800` | ## Receipt @@ -100,26 +100,25 @@ CI compares them. }, { "arm": "types", - "seconds": 0.3, + "seconds": 0.4, "status": "passed" }, { "arm": "test", - "coverage_percent": 90.58, - "seconds": 24.6, + "coverage_percent": 90.91, + "seconds": 29.4, "status": "passed", - "tests_passed": 115, - "tests_skipped": 2 + "tests_passed": 160 }, { "arm": "test-fast", - "seconds": 18.4, + "seconds": 14.8, "status": "passed", - "tests_passed": 102 + "tests_passed": 145 }, { "arm": "links", - "seconds": 1.0, + "seconds": 0.3, "status": "passed" }, { @@ -129,27 +128,27 @@ CI compares them. }, { "arm": "bandit", - "seconds": 1.0, + "seconds": 0.4, "status": "passed" }, { "arm": "build", - "seconds": 10.1, + "seconds": 3.3, "status": "passed" }, { "arm": "install", - "seconds": 22.5, + "seconds": 17.7, "status": "passed" }, { "arm": "secret", - "seconds": 23.5, + "seconds": 9.3, "status": "passed" }, { "arm": "gnn", - "seconds": 7.8, + "seconds": 0.9, "status": "passed", "tests_passed": 15 } @@ -242,13 +241,13 @@ CI compares them. }, "coverage_floor": 90, "environment": { - "lock_sha256": "ba33e645ea9521dbd6708dc830afabe681f57a6c7f4a711f3a6e956295049aaa", + "lock_sha256": "e6d9bd0a85a6ba11b554b2f73032dd0ba58ec37e11e9f7a0c36a6f03b7ca6fff", "machine": "arm64", "python": "3.12.16", "system": "Darwin", - "uv": "0.13.7" + "uv": "0.13.8" }, - "recorded": "2026-11-12T09:55:02-0800", + "recorded": "2026-11-16T13:26:10-0800", "schema": "biofirm-gate-receipt/v1" } ``` diff --git a/docs/io.md b/docs/io.md index 5ea55e7..af42a4c 100644 --- a/docs/io.md +++ b/docs/io.md @@ -49,6 +49,13 @@ checked against the configuration's variable set, `metrics` is exactly `posterior_entropy`, and every value must be finite. Read or write this format through `SimulationResult`/`validate_result_document` rather than by hand. +Validation checks initial values and zero initial controls, bounds transition +noise by three configured standard deviations, reconstructs clipped next-state +values, and recomputes posterior entropy from recorded beliefs. Action IDs must +match effective signed control using the `1e-12` hold threshold, including zero +and near-zero strengths. Artifact keys and schema version remain unchanged. +Retain rejected historical documents as evidence; do not rewrite them to pass. + Use it as: ```bash @@ -65,10 +72,14 @@ Each artifact records the request hash, prompt version, source hash, a provider-cost estimate, a human review state, and structured findings extracted verbatim from the provider response (with hedge-aware confidence). -Resume is allowed only when both files, their hashes, the prompt version and -hash, and the source hash agree. Structural validation establishes artifact -integrity; it never certifies the factual quality of citations or research -claims. Execution is split into independently rerunnable +Each complete JSON/Markdown pair is hashed and checkpointed to the manifest +before another provider job starts. A later provider or rendering failure +therefore leaves already committed pairs resumable; an incomplete pair is not a +completed manifest entry. Resume is allowed only when both files, their hashes, +the prompt version and hash, and the source hash agree. Manifest costs estimate +committed artifact entries, not billed provider usage. Structural validation +establishes artifact integrity; it never certifies factual quality of citations +or research claims. Execution remains split into independently rerunnable acquire/prepare/execute/parse/render stages. ## Where outputs belong diff --git a/src/biofirm/config.py b/src/biofirm/config.py index e6a5a3c..db526da 100644 --- a/src/biofirm/config.py +++ b/src/biofirm/config.py @@ -210,13 +210,15 @@ class RelationshipConfig: def __post_init__(self) -> None: if not isinstance(self.dependencies, Mapping): raise ConfigError("relationship dependencies must be an object") - object.__setattr__(self, "dependencies", MappingProxyType(dict(self.dependencies))) if not isinstance(self.target, str) or not self.target.strip(): raise ConfigError("relationship targets must be non-empty strings") for source, coefficient in self.dependencies.items(): if not isinstance(source, str) or not source.strip(): raise ConfigError("relationship dependency keys must be non-empty strings") _finite(coefficient, f"relationships.{self.target}.dependencies.{source}") + object.__setattr__( + self, "dependencies", MappingProxyType(dict(sorted(self.dependencies.items()))) + ) @classmethod def from_dict(cls, target: str, raw: Mapping[str, Any]) -> RelationshipConfig: @@ -350,9 +352,14 @@ def __post_init__(self) -> None: raise ConfigError("configuration.variables must be an object") if not isinstance(self.relationships, Mapping): raise ConfigError("configuration.relationships must be an object") - object.__setattr__(self, "variables", MappingProxyType(dict(self.variables))) - object.__setattr__(self, "relationships", MappingProxyType(dict(self.relationships))) self.validate() + # Hash-equivalent mappings must assign RNG draws and sums identically. + object.__setattr__( + self, "variables", MappingProxyType(dict(sorted(self.variables.items()))) + ) + object.__setattr__( + self, "relationships", MappingProxyType(dict(sorted(self.relationships.items()))) + ) @classmethod def from_dict(cls, raw: Mapping[str, Any]) -> BiofirmConfig: diff --git a/src/biofirm/controllers.py b/src/biofirm/controllers.py index 05e31c4..27a52b0 100644 --- a/src/biofirm/controllers.py +++ b/src/biofirm/controllers.py @@ -110,7 +110,7 @@ def act(self, values: Mapping[str, float]) -> ControlDecision: for name, variable in self.config.variables.items(): action = int(self._rng.integers(0, 3)) controls[name] = float((action - 1) * variable.control_strength) - action_ids[name] = action + action_ids[name] = _action_for_control(controls[name]) return ControlDecision( controls=controls, action_ids=action_ids, @@ -205,7 +205,7 @@ def act(self, values: Mapping[str, float]) -> ControlDecision: belief = np.asarray(qs[0]).reshape(-1) vfe = float(np.asarray(info["vfe"]).reshape(-1)[0]) controls[name] = float((action - 1) * variable.control_strength) - action_ids[name] = action + action_ids[name] = _action_for_control(controls[name]) beliefs[name] = tuple(float(item) for item in belief) vfes[name] = vfe efes[name] = tuple(float(item) for item in np.asarray(expected).reshape(-1)) diff --git a/src/biofirm/research.py b/src/biofirm/research.py index 17e03c8..8b4286e 100644 --- a/src/biofirm/research.py +++ b/src/biofirm/research.py @@ -472,7 +472,6 @@ def run( max_tokens=max_tokens, ) destination = Path(output_dir) - manifest_path = destination / "manifest.json" written: list[Path] = [] for job in jobs: if job.resumable: @@ -480,7 +479,6 @@ def run( response = self.execute(job.request) document = self.build_document(job, response) _atomic_json(job.target, document) - written.append(job.target) _atomic_text(job.target.with_suffix(".md"), self.render(document)) manifest_entries[job.relative_json] = _artifact_entry( destination, @@ -488,20 +486,28 @@ def run( job.source_hash, cost_usd_estimate=response.cost_usd_estimate(), ) - entries = [manifest_entries[key] for key in sorted(manifest_entries)] - _atomic_json( - manifest_path, - { - "schema_version": RESEARCH_MANIFEST_VERSION, - "provider": "perplexity", - "live": True, - "cost_usd_total": sum(_entry_cost(destination, entry) for entry in entries), - "entries": entries, - }, - ) + _write_manifest(destination, manifest_entries) + written.append(job.target) + if not written: + _write_manifest(destination, manifest_entries) return written +def _write_manifest(destination: Path, manifest_entries: Mapping[str, Mapping[str, Any]]) -> None: + """Commit only complete artifact pairs before another provider job starts.""" + entries = [manifest_entries[key] for key in sorted(manifest_entries)] + _atomic_json( + destination / "manifest.json", + { + "schema_version": RESEARCH_MANIFEST_VERSION, + "provider": "perplexity", + "live": True, + "cost_usd_total": sum(_entry_cost(destination, entry) for entry in entries), + "entries": entries, + }, + ) + + def build_prompt(region: Mapping[str, Any], persona: str) -> str: region_json = json.dumps(region, sort_keys=True, ensure_ascii=False) return ( diff --git a/src/biofirm/simulation.py b/src/biofirm/simulation.py index 1692e58..4fd60b8 100644 --- a/src/biofirm/simulation.py +++ b/src/biofirm/simulation.py @@ -3,7 +3,7 @@ from __future__ import annotations import math -from collections.abc import Mapping +from collections.abc import Mapping, Sequence from dataclasses import dataclass, field from pathlib import Path from typing import Any @@ -208,11 +208,7 @@ def _transition( source_config = config.variables[source] normalised = (float(values[source]) - source_config.target) / source_config.span relationship_effect += coefficient * normalised * variable.span - if name not in decision.controls: - raise SimulationError(f"controller omitted control for {name!r}") control = float(decision.controls[name]) - if not math.isfinite(control) or abs(control) > variable.control_strength + 1e-9: - raise SimulationError(f"controller produced invalid control for {name!r}") raw_noise = float(rng.normal(0.0, variable.noise)) noise = float(np.clip(raw_noise, -3.0 * variable.noise, 3.0 * variable.noise)) proposed = current + trend_effect + relationship_effect + control + noise @@ -304,7 +300,7 @@ def _action_id(control: float) -> int: return 1 -def _belief_entropy(values: tuple[float, ...]) -> float: +def _belief_entropy(values: Sequence[float]) -> float: """Shannon entropy of a belief distribution (0 for an empty/known state).""" if not values: return 0.0 @@ -497,12 +493,18 @@ def validate_result_document(document: Any) -> dict[str, Any]: variable = validated_config.variables[name] if not variable.minimum <= float(value) <= variable.maximum: raise ResultSchemaError(f"record {index} has out-of-bounds values.{name}") + if index == 0 and not math.isclose( + float(value), variable.initial, rel_tol=1e-9, abs_tol=1e-9 + ): + raise ResultSchemaError(f"record {index} has inconsistent initial values.{name}") expected_observation = _observation(float(value), variable) if record["observations"][name] != expected_observation: raise ResultSchemaError(f"record {index} has inconsistent observations.{name}") control = float(record["controls"][name]) if abs(control) > variable.control_strength + 1e-9: raise ResultSchemaError(f"record {index} has out-of-bounds controls.{name}") + if index == 0 and control != 0.0: + raise ResultSchemaError(f"record {index} has inconsistent initial controls.{name}") expected_action = _action_id(control) if record["action_ids"][name] != expected_action: raise ResultSchemaError(f"record {index} has inconsistent action_ids.{name}") @@ -512,10 +514,15 @@ def validate_result_document(document: Any) -> dict[str, Any]: [ abs(record["values"][name] - validated_config.variables[name].target) / validated_config.variables[name].span - for name in variable_names + for name in validated_config.variables ] ) ) + expected_entropy = float( + np.mean( + [_belief_entropy(record["beliefs"][name]) for name in validated_config.variables] + ) + ) if ( not math.isclose( float(record["metrics"]["homeostatic_error"]), @@ -535,10 +542,16 @@ def validate_result_document(document: Any) -> dict[str, Any]: rel_tol=1e-9, abs_tol=1e-9, ) + or not math.isclose( + float(record["metrics"]["posterior_entropy"]), + expected_entropy, + rel_tol=1e-9, + abs_tol=1e-9, + ) or record["metrics"]["bounded"] != 1.0 ): raise ResultSchemaError(f"record {index} has inconsistent metrics") - for name in variable_names: + for name, variable in validated_config.variables.items(): if index == 0: expected_diagnostics = { "trend": 0.0, @@ -547,20 +560,37 @@ def validate_result_document(document: Any) -> dict[str, Any]: "noise": 0.0, } else: + previous_values = records[index - 1]["values"] relationship_effect = 0.0 relationship = validated_config.relationships.get(name) if relationship: - previous_values = records[index - 1]["values"] for source, coefficient in relationship.dependencies.items(): source_config = validated_config.variables[source] normalised = ( float(previous_values[source]) - source_config.target ) / source_config.span - relationship_effect += ( - coefficient * normalised * validated_config.variables[name].span - ) + relationship_effect += coefficient * normalised * variable.span + noise = float(record["diagnostics"][f"{name}.noise"]) + if abs(noise) > 3.0 * variable.noise: + raise ResultSchemaError( + f"record {index} has out-of-bounds diagnostics.{name}.noise" + ) + proposed = ( + float(previous_values[name]) + + variable.trend + + relationship_effect + + float(record["controls"][name]) + + noise + ) + expected_value = float(np.clip(proposed, variable.minimum, variable.maximum)) + if not math.isclose( + float(record["values"][name]), expected_value, rel_tol=1e-9, abs_tol=1e-9 + ): + raise ResultSchemaError( + f"record {index} has inconsistent transition values.{name}" + ) expected_diagnostics = { - "trend": validated_config.variables[name].trend, + "trend": variable.trend, "relationship": relationship_effect, "control": float(record["controls"][name]), } @@ -607,6 +637,14 @@ def _validate_decision(config: BiofirmConfig, decision: ControlDecision) -> None for name, action in decision.action_ids.items(): if isinstance(action, bool) or not isinstance(action, int) or action not in {0, 1, 2}: raise SimulationError(f"controller produced invalid action for {name!r}") + control = decision.controls[name] + if ( + not _finite_json_number(control) + or abs(control) > config.variables[name].control_strength + 1e-9 + ): + raise SimulationError(f"controller produced invalid control for {name!r}") + if action != _action_id(control): + raise SimulationError(f"controller produced inconsistent action for {name!r}") def parameter_sweep( diff --git a/src/biofirm/text.py b/src/biofirm/text.py index 59af969..f977e8f 100644 --- a/src/biofirm/text.py +++ b/src/biofirm/text.py @@ -111,8 +111,8 @@ def process(self, description: str) -> BioregionProfile: evidence.append(span) variables: dict[str, Mapping[str, Any]] = {} ambiguities: list[tuple[int, str]] = [] - sentences = list(re.finditer(r"[^.!?]+[.!?]?", description)) - for sentence_match in sentences: + # Keep decimal points, including leading fractions, inside their sentence. + for sentence_match in re.finditer(r"(?:[^.!?]|\.(?=\d))+[.!?]?", description): sentence = sentence_match.group(0) sentence_start = sentence_match.start() aliases = [ diff --git a/tests/test_config.py b/tests/test_config.py index f920aac..c42a037 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -1,5 +1,8 @@ from __future__ import annotations +import json +from pathlib import Path + import pytest from biofirm.config import ( @@ -178,3 +181,51 @@ def test_config_parsers_reject_unknown_and_malformed_fields() -> None: base["seed"] = True with pytest.raises(ConfigError, match="seed"): BiofirmConfig.from_dict(base) + + +@pytest.mark.parametrize("controller_name", ["slide-rule", "random", "active-inference"]) +def test_hash_equivalent_mapping_orders_replay_identically( + tmp_path: Path, controller_name: str +) -> None: + from biofirm.simulation import simulate, write_result + + variables = { + name: VariableConfig( + name=name, + initial=0.5, + target=0.0, + minimum=0.0, + maximum=1.0, + noise=0.01 * (index + 1), + control_strength=0.1, + ) + for index, name in enumerate(("delta", "gamma", "beta", "alpha")) + } + relationship = RelationshipConfig("delta", {"gamma": 1.0, "beta": -1e16, "alpha": 1e16}) + config = BiofirmConfig( + "replay-order", + variables, + {"delta": relationship}, + controller=ControllerConfig( + name=controller_name, stochastic=controller_name == "active-inference" + ), + ) + reordered = BiofirmConfig( + config.name, + dict(reversed(list(config.variables.items()))), + { + "delta": RelationshipConfig( + "delta", dict(reversed(list(relationship.dependencies.items()))) + ) + }, + controller=config.controller, + ) + assert config.sha256 == reordered.sha256 + first = simulate(config, steps=2, seed=17) + assert first.records[1].diagnostics["delta.relationship"] == 0.5 + assert simulate(reordered, steps=2, seed=17).to_dict() == first.to_dict() + + artifact = write_result(first, tmp_path / "replay.json") + stored = json.loads(artifact.read_text(encoding="utf-8")) + reloaded = BiofirmConfig.from_dict(stored["config"]) + assert simulate(reloaded, steps=2, seed=17).to_dict() == stored diff --git a/tests/test_research.py b/tests/test_research.py index 16e47a9..98b786e 100644 --- a/tests/test_research.py +++ b/tests/test_research.py @@ -1,9 +1,12 @@ from __future__ import annotations +import hashlib import json import threading import time from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from typing import Any import pytest @@ -12,6 +15,7 @@ ResearchError, ResearchFinding, ResearchRequest, + ResearchResponse, ResearchService, _parse_response, build_prompt, @@ -414,3 +418,71 @@ def test_manifest_records_cost_total_findings_and_review(tmp_path, server, monke } with pytest.raises(ResearchError, match="review status"): set_review_status(document, "unknown") + + +@pytest.mark.parametrize("failure_stage", ["provider", "render"]) +def test_interrupted_research_preserves_committed_pairs_on_resume( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, failure_stage: str +) -> None: + class InterruptingProvider: + def __init__(self) -> None: + self.calls: list[str] = [] + self.resuming = False + + def complete(self, request: ResearchRequest) -> ResearchResponse: + self.calls.append(request.region_id) + if failure_stage == "provider" and request.region_id == "r2" and not self.resuming: + raise ResearchError("interrupted second job") + return ResearchResponse( + content=f"Response {len(self.calls)} for {request.region_id}.", + citations=("https://example.test/source",), + model=request.model, + usage={"prompt_tokens": 100, "completion_tokens": 50, "total_tokens": 150}, + provider_request_id=f"request-{len(self.calls)}", + ) + + provider = InterruptingProvider() + service = ResearchService(provider) + original_render = service.render + + def render(document: dict[str, Any]) -> str: + if ( + failure_stage == "render" + and document["request"]["region_id"] == "r2" + and not provider.resuming + ): + raise ResearchError("interrupted second job") + return original_render(document) + + monkeypatch.setattr(service, "render", render) + regions = [{"_id": "r1", "name": "First"}, {"_id": "r2", "name": "Second"}] + with pytest.raises(ResearchError, match="interrupted second job"): + service.run(regions, ["ecologist"], output_dir=tmp_path, live=True) + + first_json = tmp_path / "r1" / "ecologist.json" + first_markdown = first_json.with_suffix(".md") + completed_pair = (first_json.read_bytes(), first_markdown.read_bytes()) + manifest_path = tmp_path / "manifest.json" + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + assert [entry["json"] for entry in manifest["entries"]] == ["r1/ecologist.json"] + first_entry = manifest["entries"][0] + assert first_entry["json_sha256"] == hashlib.sha256(completed_pair[0]).hexdigest() + assert first_entry["markdown_sha256"] == hashlib.sha256(completed_pair[1]).hexdigest() + one_job_cost = estimate_cost({"prompt_tokens": 100, "completion_tokens": 50}) + assert manifest["cost_usd_total"] == pytest.approx(one_job_cost) + second_json = tmp_path / "r2" / "ecologist.json" + assert second_json.exists() == (failure_stage == "render") + assert not second_json.with_suffix(".md").exists() + + provider.resuming = True + written = ResearchService(provider).run(regions, ["ecologist"], output_dir=tmp_path, live=True) + assert written == [second_json] + assert provider.calls == ["r1", "r2", "r2"] + assert (first_json.read_bytes(), first_markdown.read_bytes()) == completed_pair + resumed = json.loads(manifest_path.read_text(encoding="utf-8")) + assert [entry["json"] for entry in resumed["entries"]] == [ + "r1/ecologist.json", + "r2/ecologist.json", + ] + assert resumed["entries"][0] == first_entry + assert resumed["cost_usd_total"] == pytest.approx(2.0 * one_job_cost) diff --git a/tests/test_simulation.py b/tests/test_simulation.py index 750fd7f..41546ca 100644 --- a/tests/test_simulation.py +++ b/tests/test_simulation.py @@ -1,11 +1,20 @@ from __future__ import annotations import copy +import json +import math +from collections.abc import Mapping +from pathlib import Path import pytest -from biofirm.config import load_config -from biofirm.controllers import ActiveInferenceController, RandomController, SlideRuleController +from biofirm.config import BiofirmConfig, load_config +from biofirm.controllers import ( + ActiveInferenceController, + ControlDecision, + RandomController, + SlideRuleController, +) from biofirm.simulation import ( ResultSchemaError, SimulationError, @@ -250,3 +259,138 @@ def act(self, values): # type: ignore[no-untyped-def] with pytest.raises(SimulationError, match="exactly"): simulate(config, steps=1, controller=BadController()) # type: ignore[arg-type] + + +def _single_variable_config(**overrides: float) -> BiofirmConfig: + variable = { + "initial": 5.0, + "target": 5.0, + "minimum": 0.0, + "maximum": 10.0, + "trend": 0.5, + "noise": 0.0, + "control_strength": 0.5, + } + variable.update(overrides) + return BiofirmConfig.from_dict({"name": "artifact-validation", "variables": {"x": variable}}) + + +@pytest.mark.parametrize("noise", [0.0, 0.2]) +def test_result_validator_rejects_noise_outside_three_sigma(noise: float) -> None: + document = simulate( + _single_variable_config(noise=noise), steps=1, seed=3, controller_name="slide-rule" + ).to_dict() + document["records"][1]["diagnostics"]["x.noise"] = 1e-300 if noise == 0.0 else 3.0 * noise + 0.1 + with pytest.raises(ResultSchemaError, match="out-of-bounds diagnostics"): + validate_result_document(document) + + +def test_result_validator_rejects_unreconstructable_transition() -> None: + document = simulate( + _single_variable_config(), steps=1, seed=3, controller_name="slide-rule" + ).to_dict() + record = document["records"][1] + record["values"]["x"] = 6.0 + record["observations"]["x"] = 2 + record["metrics"]["homeostatic_error"] = 0.1 + record["metrics"]["homeostatic_score"] = 0.9 + document["summary"]["final_homeostatic_error"] = 0.1 + with pytest.raises(ResultSchemaError, match="inconsistent transition"): + validate_result_document(document) + + +@pytest.mark.parametrize("component", ["values", "controls"]) +def test_result_validator_rejects_inconsistent_initial_record(component: str) -> None: + document = simulate(_single_variable_config(), steps=0, controller_name="slide-rule").to_dict() + record = document["records"][0] + if component == "values": + record["values"]["x"] = 6.0 + record["observations"]["x"] = 2 + record["metrics"]["homeostatic_error"] = 0.1 + record["metrics"]["homeostatic_score"] = 0.9 + document["summary"]["final_homeostatic_error"] = 0.1 + else: + record["controls"]["x"] = 0.25 + record["action_ids"]["x"] = 2 + record["metrics"]["control_effort"] = 0.25 + document["summary"]["total_control_effort"] = 0.25 + with pytest.raises(ResultSchemaError, match=f"inconsistent initial {component}"): + validate_result_document(document) + + +@pytest.mark.parametrize("entropy", [-1.0, 0.0, math.log(2.0) + 0.25]) +def test_result_validator_recomputes_posterior_entropy(entropy: float) -> None: + document = simulate( + _single_variable_config(), steps=1, seed=3, controller_name="slide-rule" + ).to_dict() + record = document["records"][1] + record["beliefs"]["x"] = [0.5, 0.5, 0.0] + record["metrics"]["posterior_entropy"] = entropy + with pytest.raises(ResultSchemaError, match="inconsistent metrics"): + validate_result_document(document) + + +@pytest.mark.parametrize(("initial", "trend", "expected"), [(9.0, 3.0, 10.0), (1.0, -3.0, 0.0)]) +def test_clipped_transition_artifacts_round_trip( + tmp_path: Path, initial: float, trend: float, expected: float +) -> None: + result = simulate( + _single_variable_config(initial=initial, target=initial, trend=trend), + steps=1, + controller_name="slide-rule", + ) + path = write_result(result, tmp_path / "clipped.json") + document = json.loads(path.read_text(encoding="utf-8")) + assert document["records"][1]["values"]["x"] == expected + assert document["records"][1]["diagnostics"]["x.trend"] == trend + assert document == result.to_dict() + + +@pytest.mark.parametrize( + "controller_name", ["slide-rule", "random", "active-inference", "business"] +) +@pytest.mark.parametrize("control_strength", [0.0, 1e-13, 1e-12]) +def test_zero_strength_action_contract_round_trips( + tmp_path: Path, controller_name: str, control_strength: float +) -> None: + raw = load_config().to_dict() + for variable in raw["variables"].values(): + variable["control_strength"] = control_strength + config = BiofirmConfig.from_dict(raw) + result = simulate(config, steps=2, seed=2, controller_name=controller_name) + for record in result.records: + assert all(abs(control) <= control_strength for control in record.controls.values()) + assert all(action == 1 for action in record.action_ids.values()) + path = write_result(result, tmp_path / "zero-strength.json") + assert json.loads(path.read_text(encoding="utf-8")) == result.to_dict() + + +@pytest.mark.parametrize( + ("control", "action", "message"), + [ + (0.0, 0, "inconsistent action"), + (0.0, 2, "inconsistent action"), + (1e-13, 2, "inconsistent action"), + (-1e-13, 0, "inconsistent action"), + (0.25, 0, "inconsistent action"), + (-0.25, 2, "inconsistent action"), + (float("nan"), 1, "invalid control"), + (float("inf"), 2, "invalid control"), + (0.6, 2, "invalid control"), + ], +) +def test_simulate_rejects_inconsistent_decision_before_success( + control: float, action: int, message: str +) -> None: + class FixedController: + def reset(self, seed: int) -> None: + del seed + + def act(self, values: Mapping[str, float]) -> ControlDecision: + return ControlDecision( + controls={name: control for name in values}, + action_ids={name: action for name in values}, + ) + + with pytest.raises(SimulationError, match=message): + simulate(_single_variable_config(), steps=1, controller=FixedController()) diff --git a/tests/test_text_business.py b/tests/test_text_business.py index 66f6244..e43a4b1 100644 --- a/tests/test_text_business.py +++ b/tests/test_text_business.py @@ -165,3 +165,50 @@ def test_business_action_id_matches_simulation_contract() -> None: for control in (0.0, 1e-13, -1e-13, 1.5, -1.5): assert business._action_id(control) == simulation._action_id(control) + + +@pytest.mark.parametrize( + ("range_text", "minimum", "maximum"), + [ + ("between 10.5 and 20.5 index", 10.5, 20.5), + ("between -10.5 and +20.5 index", -10.5, 20.5), + ("between -.5 and +.75 index", -0.5, 0.75), + ("-10.5--2.5 index", -10.5, -2.5), + ("between 20.5 and 10.5 index", 10.5, 20.5), + ], +) +def test_decimal_ranges_preserve_values_units_and_original_offsets( + range_text: str, minimum: float, maximum: float +) -> None: + description = ( + f"A river. Biodiversity {range_text}. Soil moisture 30.25 to 40.75 %! Another sentence." + ) + profile = BioregionTextProcessor(load_config()).process(description) + assert profile.variables["biodiversity"]["minimum"] == minimum + assert profile.variables["biodiversity"]["maximum"] == maximum + assert profile.variables["biodiversity"]["unit"] == "index" + assert profile.variables["soil_moisture"]["minimum"] == 30.25 + assert profile.variables["soil_moisture"]["maximum"] == 40.75 + assert profile.variables["soil_moisture"]["unit"] == "%" + spans = {span.value: span for span in profile.evidence if span.category == "variable-range"} + span = spans["biodiversity"] + assert span.start == description.index(range_text) + assert span.end == span.start + len(range_text) + assert description[span.start : span.end] == range_text + assert span.text == f"Biodiversity {range_text}." + assert not profile.ambiguities + + +def test_decimal_range_attribution_remains_ambiguous_when_nearly_equidistant() -> None: + description = "soil_moisture 40.25 to 50.75 biodiversity" + profile = BioregionTextProcessor(load_config()).process(description) + assert profile.variables == {} + assert profile.ambiguities == ((14, "40.25 to 50.75 biodiversity"),) + + +def test_decimal_range_does_not_borrow_an_alias_from_another_sentence() -> None: + profile = BioregionTextProcessor(load_config()).process( + "Biodiversity is unmeasured. between 10.5 and 20.5 index." + ) + assert profile.variables == {} + assert not [span for span in profile.evidence if span.category == "variable-range"] diff --git a/tools/gate_report.py b/tools/gate_report.py index 1015888..559b4b3 100644 --- a/tools/gate_report.py +++ b/tools/gate_report.py @@ -223,8 +223,8 @@ def render(data: dict[str, Any]) -> str: which the `gnn` arm owns. - GNN conformance (`gnn`): **{gnn.get("tests_passed", "unrecorded")} passed**. The full suite reports {full.get("tests_skipped", 0)} skips from - `tests/test_gnn.py`: its pinned-parser checks need the export the `gnn` arm - writes, so they skip there and run here. + `tests/test_gnn.py`. Its pinned-parser checks run when a prepared export is + available, otherwise they skip; the separate `gnn` arm requires that export. The preserved pre-gate baseline reported **{baseline_test.get("tests_passed", "unrecorded")} passed** at