Conversation
T0 fixes how numbers will be judged before any number exists. It binds the step-7 training collection, the step-8 fit and #263-B alike, because a rule chosen after a look at training data is a preference with a timestamp. Frozen here: the pairing unit is the cell (rung, workload, regime) and the statistic is the median paired log-ratio, with index pairing named and forbidden — the harness shuffles cells and runs a cell's N iterations back to back, so same-index iterations of the two engines have no temporal correspondence. The two-margin form of acceptance, the four-state decision automaton, invalidation and retry semantics, and host selection and replacement rules are frozen. Training may derive N and nothing else. Not frozen, and recorded as typed slots rather than filled in: M, A, their scope, the caps K and B, the null-primary-metric outcome, the uncertainty model and its parameters, the retry budget, and five host-predicate slots. A drafter who fills one of those has replaced the owner. One conflict is open and cannot be settled inside T0: only two of #262's seven gate phases yield a standalone gateable number, so any primary/diagnostic split changes #262's acceptance surface and needs ratification against it. NORMATIVE_RATIFICATION_REQUIRED_AGAINST_262. Hostile review leaves five "yes" answers, every one of them an empty slot or that conflict. T0_NOT_FREEZABLE, collection_authorized: false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
… open R1 — the user-visible path is `launcher-e2e`, evaluated as two preregistered regimes, cold and warm, both primary strata of one surface and never collapsed into one number. `core-usage` stays a published startup/floor baseline and does not stand in for the gate. R2 — peak RSS remains a required resource gate where a mechanism exists. Allocation/heap profiling is diagnostic, published where practical; a missing allocation profile is neither INVALID nor FAIL, and no instrumentation is added to obtain one. R3 — the #262 ratification blocker is withdrawn as a misreading. #262 asks for published BASELINES and states its gate once — an explicit budget and no material regression of the user-visible path — while #263 requires separate measurement only "where possible" and demands unavailable stages be marked. A phase published as composed and marked unavailable is that method working. Every stage baseline now carries one of four honest classes, and a derived number may not be renamed into something stronger than it proves. R4 — Option 2. K, B and caps_apply_to are REMOVED from the contract rather than filled: per-phase caps are an acceptance policy #262 does not ask for, and they would turn profiling surfaces back into hidden vetoes. A guardrail, if a real profile ever justifies one, arrives as its own amendment. The null-primary-metric switch is replaced by three distinct states: no mechanism before collection makes the session ineligible so it never starts; a metric that goes null mid-attempt invalidates that attempt; exhausting the retry budget yields NO_DECISION. PASS remains structurally unreachable in all three. T0-5 chooses the deterministic gray zone over a bootstrap across cells: the cells are fixed preregistered acceptance strata, not an IID sample, and resampling them would lend a workload list statistical authority it has not earned. The limits themselves stay unresolved. Twelve slots remain, all of them budget numbers or host-predicate fields. Verdict: T0_SKELETON_READY_FOR_NUMERIC_RULINGS. Status stays NOT_FROZEN and collection_authorized: false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh
…dimension Two structural clarifications before any number is chosen. No harness change, no value selected, no new slot. The regime identifier is the instrument's: `process-cold`, not a bare `cold`. `warm` is still a fresh process, after discarded warmup iterations, with the OS and filesystem caches warm. `machine-cold` is not claimed by the instrument, so it is not a regime, not a gate and not an alias here. Documentation that says `cold` while evidence arrives as `process-cold` buys a three-day argument later about whether one is an alias of the other. Peak RSS carries the regime dimension too. `peak_rss_bytes` is captured on every measured cell and a cell's key includes its regime, so memory exists separately for `process-cold` and for `warm` rather than as one number beside the surface. The primary set is therefore FOUR independent gates — elapsed time and peak RSS, each in each regime — with no compensation between regimes and none between time and memory. This does not restore per-phase vetoes: all four sit on the same user-visible surface, and memory on a diagnostic rung stays published and never gating. T0-2 gains the dimensional constraint that follows: `A` carries the gate's unit, the primary set now spans milliseconds and bytes, and one absolute value cannot serve both. "One pair for all gates" is not a budget waiting to be chosen for `A`; it is not a quantity. Recorded as a constraint on the coming numeric ruling — `M_A_scope` stays unresolved and no value is picked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh
…t values T0-2 defined two coordinates and T0-5 then judged a scalar `value` against a PASS_LIMIT. Nothing produced that scalar and nothing said which number it was — relative, absolute, some normalisation, the larger of the two. The decision function was underdetermined, and the missing piece was a dimension, not a limit. R5 — the gray zone is two-dimensional. PASS iff relative <= M_pass OR absolute <= A_pass; FAIL iff relative >= M_fail AND absolute >= A_fail; otherwise NO_DECISION, with M_pass < M_fail and A_pass < A_fail required. The two conditions are then mutually exclusive, so each gate is a total single-valued function. PASS_LIMIT, FAIL_LIMIT and gray_zone_margin_binding are removed as symptoms of the wrong model, not filled in. R6/R7 — eight budgets, per resource, superseding M, A and M_A_scope: time gates 5 %/50 ms to pass, 10 %/100 ms to fail; RSS gates 10 %/32 MiB to pass, 20 %/64 MiB to fail, recorded in bytes so a checker needs no arithmetic. Both regimes of a resource share its budgets; time and memory share nothing, because one absolute value cannot span milliseconds and bytes. R8 — retry_budget: 1. One initial attempt plus at most one full-session retry after INVALID; a second INVALID is NO_DECISION. R9 — a fresh manifest per session, identity rechecked after it, drift INVALID. R10-R13 — the host predicate splits declared provisioning evidence from runtime-verified invariants, and does not call the first a proof: a guest OS cannot establish what the hypervisor is doing. Power policy is pinned per platform with turbo left alone, background prohibitions name transient heavy jobs rather than a debloat ritual, and quiesce is 120 s with a measured 60 s window — whose failure is "not eligible to start", never INVALID, so it starts no clock, produces no number and cannot be outcome-selective. Zero UNRESOLVED_OWNER_DECISION slots remain. Status stays NOT_FROZEN and collection_authorized: false until the hostile freeze review is done — completeness is what makes a freeze reviewable, not the freeze itself. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh
…gates Phase 3, in one commit, on the four findings the freeze review left plus the platform split that #355 made necessary. S1 — the gate population belongs to T0, not to a function. The gates are computed over exactly the decisive workloads of the D7-bound manifest that the `launcher-e2e` rung admits — all thirteen at the current manifest — and calibration workloads never enter a cutover gate. A missing decisive workload, a missing engine side, an unexpected extra cell or a calibration cell inside the set each make the attempt INVALID: a missing cell is not a smaller denominator. The instrument's applicability rule implements that sentence; it is not the source of it, because two readers must not be able to answer differently about which cells a gate covers. S2 — the numeric domain is checked BEFORE the two-dimensional rule, not inside it. Every quantity a gate consumes must be defined, real and finite, with medians strictly positive; None, NaN or an infinity in R or either coordinate invalidates the attempt. Checked first because the rule's OR would otherwise let `relative = NaN, absolute = 10 ms` pass on the strength of the other coordinate. A comparison against an undefined number is not one that succeeded. S3 — a primary cell holds exactly N samples, each finite, positive and carrying its stratum's declared metric kind. One null, one missing sample or one unexpected metric kind invalidates the session. A median over the surviving subset is forbidden by name: the instrument's summariser drops absent memory samples silently, so without this rule a cell that lost half its measurements reports a confident median of the other half. S8 — since the memory quantity is platform-local, gate identity carries its stratum: eight primary gates, four per stratum, with `max_process_peak_resident` on Linux and `max_process_peak_commit` on Windows. `resident == commit` is asserted nowhere, the strata are never pooled, and the two memory budget families carry identical numbers while remaining two budgets — they agree today because both came from one product budget before any data existed, and either may move without the other. S7 — the replacement conditions are closed at six, with no "other environmental reason" escape hatch, and the retry budget belongs to the STRATUM. Replacement does not reset it; a valid outcome-bearing session closes the stratum and is retained forever; re-measurement requires a previous INVALID and remaining budget. Every replacement records both binding identities and the budget left, so a reader can see what it did and did not buy. Verified by enumeration rather than by reading: the eight-gate roll-up is total and single-valued over all 6561 combinations, 108 boundary points across the three budget families yield exactly one verdict each, and the domain check rejects NaN and both infinities before the rule sees them. Stale vocabulary swept from the normative text: no generic "peak RSS" gate, no "four primary gates", no cross-platform R. The old names survive only where they are explicitly named obsolete. Status is unchanged and deliberately so: NOT_FROZEN, collection_authorized false. This closes the content; the freeze is its own review and its own commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh
…ent one Three findings from the Phase-3 review, each a sentence. F1 — T0-1 cited `562a7f7232da` as the identity steps 4, 5 and 6 were accepted on. After the S8 memory-semantics repair that is no longer true: they were re-bound to `104c384d01bf6060bdec1e7c916053ddb04b97fcbd0b39f8a4fc57b8f139672f`, which is now named in full. The superseded value is not kept as decoration — and the number is not removed either, because deleting a SHA to settle a mismatch would resolve it by deleting the checkability, which is the one thing this track is for. F2 — the pairing key now names the stratum: `(stratum, rung, workload, regime)`. Gates became platform-qualified in the previous commit while the definition of a cell still did not mention a platform, so a reader could have keyed on rung, workload and regime alone and formed one `R` across Linux and Windows — exactly the second admissible reading of one evidence set that S1 and S8 exist to prevent. Pairing never crosses a stratum, `median_c` is computed inside one gate's stratum, and no `R` spans the two. Every later use of cell, matched pair, the gate's cell set, `median_c` and `R` inherits that key, and no second term for it is introduced. F3 — "both sets above" became "all three budget families above": time, Linux resident and Windows commit. No other change. Status remains NOT_FROZEN, collection_authorized false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh
A cross-artifact conflict the Phase-3 checks did not look for: T0 judged `D = median_c(d_c)` over a gate's whole workload population, while the frozen instrument requires a preregistered decision per `(phase, workload_id, platform, regime)` cell and then rolls up `cell -> workload_class -> phase -> overall_g3`. Internal consistency was never the problem; agreement with the accepted D7 gate was. The population median did two wrong things at once. It **hid a casualty** — one decisive workload degraded past the fail margin vanishes behind eleven healthy ones — and it **double-weighted an alias**: `large-solution-control` is `alias_of` `oss-ShareX.sln`, the same path at the same pin, declared an alias precisely so the pair is never counted twice in a denominator. The instrument already resolves aliases and refuses a duplicate canonical identity by name; T0 was the artifact out of step. So the leaf is the cell. Per cell: `relative = median(Rust)/median(Python) - 1`, `absolute = median(Rust) - median(Python)`, judged by the unchanged R5 rule against its family's unchanged R6/R7 budgets — no budget moves in this commit. The eight stratum x regime x resource combinations become **gate classes**, and the roll-up is the same operator twice: cell verdicts to a class verdict, class verdicts to overall. One failing workload fails its class however many pass. `INVALID` is not a roll-up verdict at either level; admissibility is settled before a roll-up exists. Population is now counted in canonical identities rather than manifest entries — 13 decisive entries resolve to 12 canonical identities today, read from the frozen artifacts rather than asserted — and a duplicate canonical identity joins the invalidation list. Recorded so nobody has to infer it: the time classes are the `end-to-end-csharp` phase; memory is not a phase at all but each cell's `rss_policy`; the other seven D7 phases carry an explicit `not_applicable` with a reason, which is how a diagnostic surface stays non-gating while D7 completeness is still satisfied. Nothing here asks `perf_baseline.py` to change, so the harness digest does not move. Checked by enumeration rather than by reading: a class over 12 cells is total and single-valued across all 531441 combinations, the overall roll-up over 8 class verdicts across all 6561, verdicts are invariant under 20000 random reorderings of the workloads, one FAIL makes its class FAIL, one NO_DECISION makes it NO_DECISION, and `expected_d7_cells()` contains the 12 canonical ids with the alias absent. Status unchanged: NOT_FROZEN, collection_authorized false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh
G3 — the accepted verifier checks that `rollups` carries `workload_class`, `phase` and `overall_g3`, each present and non-empty, and `_present()` inspects "presence and container shape only… nothing compares, orders, or records a value". Correct for a gate forbidden to read a threshold, and it means several different reductions would pass the same check. No accepted artifact says what the three levels mean. T0 said "compatible" and left it there, which is the structural-green, semantics-unproved failure this whole track keeps catching. So the mapping is written down: `workload_class` is the reduction over the canonical workload leaf verdicts of one (platform, regime, resource); `phase` is the gating `end-to-end-csharp`, with the seven diagnostic phases carried as `not_applicable` and contributing no verdict at any level; `overall_g3` is the same operator over what remains, which is T0's own roll-up over the eight classes. And the equality is proved, not asserted. The operator is `max` under PASS < NO_DECISION < FAIL, so it is associative, commutative and idempotent: the result depends on the SET of leaf verdicts and not on grouping, ordering or nesting, and the serialized three-level reduction is therefore the same function as the two-level one. The proof is scoped in the text to discrete verdicts only — no elapsed or memory number is ever pooled across workloads, regimes, platforms or resources. G3b — one D7 cell has one set of rule keys while this contract produces two independent verdicts on it. Their location is now fixed rather than left to a later payload author: `bound`, `pass_fail_rule`, `inconclusive_band` and `comparison_statistic` each carry an `elapsed` and a `memory` sub-object, `rss_policy` names the platform-local metric, `allocation_policy` stays diagnostic. `_present` accepts a non-empty object, so this needs no schema change, no instrument change, and the harness digest does not move. Neither resource can compensate the other, and a NO_DECISION propagates unless the other is FAIL — which is the operator, not a special case. G1 — the normative rule in T0-5 and its mutual-exclusion argument now name `relative_regression_c` and `absolute_regression_c`. No unsubscripted aggregate name survives anywhere in the contract. G2 — the completion table no longer says "eight platform-qualified gates"; it lists the classes, the per-canonical-workload leaf decision, both roll-ups and the D7 serialization, each against its section. Checked: 9 resource pairs exhaustively (no compensation in either direction), 6561 class combinations for grouping and ordering invariance, 20000 random 8 x 12 verdict trees where the T0 roll-up, the D7 serialization and a flat reduction agree every time, 13 manifest entries resolving to 12 canonical identities with the alias absent from `expected_d7_cells()`. Status unchanged: NOT_FROZEN, collection_authorized false. No freeze commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
status: NOT_FROZEN·collection_authorized: false— deliberately. Content is complete; the freeze is its own review and its own commit.Stacked on #353 (T0-0), which revokes the automatic step-7 collection authority this document then gates.
The decision function
Per gate, two coordinates and no synthetic scalar:
The leaf is the cell — one canonical workload, on one stratum, in one regime, for one resource — and the same operator rolls up twice:
One failing workload fails its class however many others pass.
INVALIDis not a roll-up verdict at either level: admissibility is settled before a roll-up exists.Checked by enumeration at both levels: a class over 12 cells is total and single-valued across all 531441 combinations, and the overall roll-up over 8 class verdicts across all 6561. The 6561 figure is the second level only — it is not a check of the whole primary decision space, which has 8 x 12 = 96 leaves.
The eight primary gate classes
They are classes, not leaves: each covers the canonical decisive workloads, and the acceptance decision happens per workload cell inside it. Identity carries its stratum, because since #355 the memory quantity is platform-local. Four gates per stratum;
U_linuxandU_windowsare never pooled and no ratio is formed across them.The budgets — by resource, not by regime
Each family below applies to both regimes of its resource. The pass pair and the fail pair belong to the same gate: no regime owns one of them, and
process-colddoes not get "the pass budget" whilewarmgets "the fail budget".M_passA_passM_failA_failmax_process_peak_residentmax_process_peak_commitThe two memory families carry identical numbers and are not the same budget: resident bytes and committed bytes are different physical quantities, priced alike today because both came from one product budget before any data existed. Either may move without the other.
resident == commitis asserted nowhere.No compensation in any direction — not between regimes, not between time and memory, not between strata.
The population is owned here
Gates are computed over exactly the decisive workloads of the D7-bound manifest that the
launcher-e2erung admits, counted as canonical identities: 13 manifest entries resolve to 12 canonical identities, becauselarge-solution-controlisalias_ofoss-ShareX.sln— the same path at the same pin — and an alias earns no second vote. Calibration workloads never enter a cutover gate. A missing canonical identity, a missing engine side, an unexpected extra cell, a duplicate canonical identity or a calibration cell inside the set each make the attemptINVALID: a missing cell is not a smaller denominator. The instrument's applicability rule implements this sentence rather than being its source, so two readers cannot answer differently about which cells a gate covers.How this lands in the accepted D7 payload
D7_CELL_DIMENSIONS = (phase, workload_id, platform, regime)andD7_ROLLUP_LEVELS = (workload_class, phase, overall_g3)— 8 phases x 12 canonical workloads x 2 platforms x 2 regimes = 384 cells, each needing a rule or an explicitnot_applicable. The time classes are theend-to-end-csharpphase; memory is not a phase but each cell'srss_policy; the seven non-gating phases carrynot_applicablewith a reason, so a diagnostic surface stays non-gating and completeness is still satisfied. Nothing here asksperf_baseline.pyto change, so the harness digest does not move.Numeric domain and sample completeness
Every quantity a gate consumes must be defined, real and finite, with medians strictly positive, and this is checked before the rule — otherwise the
ORwould letrelative = NaN, absolute = 10 msthrough on the strength of the other coordinate.None,NaNor an infinity anywhere inRor either coordinate makes the attemptINVALID.A primary cell holds exactly
Nsamples, each finite, positive and carrying its stratum's declared metric kind. One null, one missing sample or one unexpected metric kind invalidates the session; a median over the surviving subset is forbidden by name, because the instrument's summariser drops absent memory samples silently.Replacement and retry continuity
Six replacement conditions, closed — no "other environmental reason". A performance outcome is never one of them. The retry budget belongs to the stratum, not the host: replacement does not reset it, a valid outcome-bearing session closes the stratum and is retained forever, and re-measurement requires a previous
INVALIDand remaining budget. A new binding starts a campaign beside the old one and erases nothing.Also resolved earlier in this PR
T0-1 pairing by cell with index pairing forbidden · T0-5 deterministic two-dimensional gray zone, not a bootstrap across cells · T0-6 session as the unit of invalidation,
retry_budget: 1, fresh manifest per session · T0-7 host predicate in two evidence classes, AC and DC on Windows, a really-waited 120 s quiesce · T0-9 training derivesNand nothing else.Hostile review
The hostile-review questions are in the document; each is answered there with the clause that closes it. None remains open that a decision could close — the one permitted item, a repeated quiesce attempt, produces no evidence and so cannot select an outcome.
🤖 Generated with Claude Code
https://claude.ai/code/session_018xhcg5opoFbSTdYHpkSXCh