Skip to content

Require policyengine-core 3.32.8 and test uprated inputs supplied only before 2015 - #9650

Merged
MaxGhenis merged 1 commit into
mainfrom
core-uprating-undefined-index
Sep 28, 2026
Merged

MaxGhenis merged 1 commit into
mainfrom
core-uprating-undefined-index

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Problem

An uprated input supplied only for a year before FIRST_MODELED_YEAR (2015) fails as soon as a later year reads it. CountryTaxBenefitSystem backdates parameters only to 2015-01-01, so the uprating index has no value at, say, 2013-01-01. policyengine-core then computed index(2015) / None:

from policyengine_us import Simulation
sim = Simulation(situation={"people": {"p": {"age": {2015: 66}, "employment_income": {2015: 30000}, "tax_exempt_interest_income": {2013: 5000}}}, "households": {"h": {"members": ["p"], "state_code": {2015: "TX"}}}})
sim.calculate("income_tax", 2015)
# TypeError: unsupported operand type(s) for /: 'float' and 'NoneType'

#9626 flagged this as a separate, pre-existing follow-up.

Fix

The fix is in core: PolicyEngine/policyengine-core#551, released in policyengine-core 3.32.8. Where an uprating index has no value, uprating now holds it flat, so an input supplied for 2013 carries over unchanged to 2015 and is uprated from there. That matches how backdate_parameters already extends each parameter's earliest value back to 2015. The same core PR also fixes base-period selection when a later known period was stored first. This PR adds the policyengine-us regression tests and requires that release.

Affected variables

312 of the 365 variables that have uprating at runtime. All are inputs, and each uses an index that starts at the 2015 floor:

  • Default AGI-per-capita series: 219. 196 of these are float USD inputs that add_default_uprating covers implicitly; the rest are declared or listed in INPUT_VARIABLES.
  • IRS SOI per-capita series: 73. Employment, pensions, capital gains, dividends, interest, alimony, farm, rental, unemployment and others.
  • CMS MOOP per capita: 18.
  • Census population: 2.
  • SSA COLA: 1.

The other 53 use CPI-U, which starts in 1913, and are unaffected. A grep for uprating = in policyengine_us/variables finds only 146 files. It misses the 219 variables that get uprating from add_default_uprating, and it shows the pre-rewrite path for the 72 that add_per_capita_uprating repoints to a _per_capita series. The test builds the list from the loaded system, not from grep.

Tests

policyengine_us/tests/core/test_pre_first_modeled_year_uprated_inputs.py (new):

  • Differential over every affected variable. Variables are grouped by where they're defined: 36 states, in_nyc, and unconditional, for 38 groups. Each group's household has its state (and in_nyc) set for every year the test reads. Geography has formulas, so it isn't carried forward. For each, an input supplied only for 2013 (or 2014-12 for the 4 monthly variables):
    • equals the same value in 2015, carried over unchanged;
    • in 2020, is nonzero, which proves the variable is defined for that household, and equals the result of supplying the same input for 2015.
  • The reported household. income_tax and household_net_income for 2015 match the same household with the interest supplied for 2015.

Against core 3.32.7's simulation.py, 39 of 40 cases fail with the reported TypeError; only the affected-set check, which runs no simulation, passes. With 3.32.8, 40 of 40 pass.

Dependency

This PR raises the floor to policyengine-core>=3.32.8, the release containing PolicyEngine/policyengine-core#551. uv.lock was relocked with uv 0.12.13, the version CI pins, and only the policyengine-core entry changes. With uv 0.12.13, release_lock.py --committed and --rehearse both pass locally.

Core 3.30.2 → 3.32.8 also picks up the 3.31.x–3.32.7 releases. pe-us PR CI already tests against the newest core, since it runs uv lock --upgrade-package policyengine-core before its test jobs, so recent PRs have been tested on those releases.

Before core merged, an earlier revision of this PR pinned core to the #551 commit, and the full suite passed against it: every baseline, contrib, microsimulation, partner and rest job.

Review

An independent Opus 5.5 review (Subfleet, code reading only) requested changes. I checked each finding by running it, then fixed it:

  • Wrong uv version. The lock was first written with uv 0.11.7. Its marker-only drift failed CI's registry-lock check, so no suite ran. I relocked with uv 0.12.13.
  • Vacuous 2020 comparison. Geography was set only for 2015. state_code for 2020 fell back to CA, so the comparison looked at masked zeros for every non-CA state group. It now sets geography for both years and asserts the 2020 value is nonzero. With the old setup, the guard fails 36 groups: every non-CA state plus in_nyc.
  • Predicate date. The affected-set check now uses the earliest input date, 2013-01-01. The set stays at 312.
  • Test count. The body said 37 states; it's 36.

axiom: n/a: engine fix to variable uprating in policyengine-core plus regression tests; no policy rule changes

🤖 Generated with Claude Code

@MaxGhenis
MaxGhenis force-pushed the core-uprating-undefined-index branch from e97d4fc to ae49ff2 Compare September 27, 2026 19:14
@MaxGhenis MaxGhenis changed the title Test uprated inputs supplied only before 2015 (needs policyengine-core#551) Require policyengine-core 3.32.8 and test uprated inputs supplied only before 2015 Sep 27, 2026
@MaxGhenis
MaxGhenis marked this pull request as ready for review September 27, 2026 19:14
policyengine-core divided by a missing uprating-index value whenever a
later year read an uprated input supplied only for a year before 2015,
raising TypeError. policyengine-core 3.32.8 (PolicyEngine/policyengine-core#551)
holds the index flat where it has no value. Require it, and cover every
affected variable (312, grouped by where each is defined) with a
differential check against the same input supplied for 2015, plus the
reported household.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis force-pushed the core-uprating-undefined-index branch from ae49ff2 to dfca154 Compare September 28, 2026 00:54
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merge audit (2026-09-28 open-PR sweep): gh pr checks exit 0 at head dfca154; mergeable=MERGEABLE; not draft; no CHANGES_REQUESTED. Independent review: GPT-6 Astra via Subfleet (job13), verdict APPROVE: All 35 checks pass; MERGEABLE, isDraft=false, reviewDecision=REVIEW_REQUIRED.; Full diff reviewed: dependency floor and lock agree on core 3.32.8; its local release tag contains the required core #551 fix.; Regression tests cover affected inputs, preserve geography across calculation years, guard against masked-zero comparisons and exercise the reported household.; No blocking defect, secret or debug leftover found; routine maintenance requires no personal decision from Max.; Not superseded: rec. Repo rules: All checks must pass; formatting and changelog required. Engine maintenance qualifies for axiom n/a. Squash is allowed and matches recent merges.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Admin-merging: the only block is the required approving review, which Max cannot give on his own PR. gh pr checks exits 0, the PR is MERGEABLE, and the independent review above is APPROVE. Pinned to head dfca154.

@MaxGhenis
MaxGhenis merged commit 39bf168 into main Sep 28, 2026
35 checks passed
@MaxGhenis
MaxGhenis deleted the core-uprating-undefined-index branch September 28, 2026 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant