Skip to content

Faster affects_comparison with an exact numeric check - #65

Open
prawalpokharel wants to merge 3 commits into
Accelergy-Project:mainfrom
prawalpokharel:perf/affects-comparison-shortcut
Open

prawalpokharel wants to merge 3 commits into
Accelergy-Project:mainfrom
prawalpokharel:perf/affects-comparison-shortcut

Conversation

@prawalpokharel

@prawalpokharel prawalpokharel commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Refs #46.

affects_comparison decides whether a symbol can change a comparison (the "dropping symbols" step in section 5.4.1 of the TCM paper). It does this by expanding f(E_2) - f(E) > 0 and f(E_2) / f(E) > 1 and checking whether s survives. When I profiled the 46 regression cases, this expansion was the largest single cost in building the pruning criteria, and most calls ended up True.

This PR looks for a witness before expanding. _has_witness_that_s_affects_comparison evaluates f exactly at a few sets of prime values, with Floats read as exact rationals. It checks whether changing only s changes f(E_2) - f(E), and whether it changes f(E_2) / f(E). If both change, s has to survive both expansions, and neither relational can reduce to True or False. So the answer is True for every input, not just the values tried. The docstring gives the full argument. If no witness turns up, the existing symbolic check runs unchanged, so the search never decides False.

Results against main at 8acf39e (serial mapper, all 46 regression cases, 4 runs per case with main and this branch back to back in the same container, alternating which goes first):

Result
Pmapping tables and joined mappings, every run identical to main
Calls also checked by the symbolic code 26,954 (15,735 decided by the witness search), 0 disagreements
Mapper time, whole suite about 15% less (median 15.5%, range 12.2-15.7% across the 4 runs)
Slowest case (TPU-v4i, GPT-3 175B, fused) 1.30x
Architecture Fused cases Unfused cases
TPU-v4i 1.25x 1.16x
Simba 1.21x 1.07x
Eyeriss 1.13x 0.99x (within run-to-run noise, 116 calls)
Simple 1.02x never calls it
Snowcat - never calls it

tests/test_affects_comparison.py and tests/test_mapper.py pass on this branch on top of current main.

Caveats:

  • Floats are compared as exact rationals. If expand rounds a term in s to exactly zero, the witness search keeps s where the full check would drop it. That means less pruning, not a wrong comparison, and it never came up on the regression suite.
  • I only timed the serial mapper.

Evidence and scripts: https://gist.github.com/prawalpokharel/98b478447332b06e067227c67973f62b

(I used Claude Code for the profiling and the implementation, and I've checked and can explain every line.)

@tanner-andrulis

Copy link
Copy Markdown
Contributor

Thanks for the PR (both this and the other already-merged one!)

While this may work in practice, it can only prove desired properties of the relational at specific sampled points. Any solutions must be proven to work generally, which is why we perform all of the reasoning symbolically rather than doing any sampling.

@prawalpokharel

Copy link
Copy Markdown
Contributor Author

Thanks! Agreed that it has to hold for every input, including the ones I didn't test. I think it does, but I explained it badly by leading with the benchmark numbers.

The fast path only ever answers True. If it can't show that s matters, the existing symbolic check runs as before. So what matters is whether a True from it is always right, and I think it is:

  1. affects_comparison returns True exactly when s is still a free symbol after expanding both f(E_2) - f(E) > 0 and f(E_2) / f(E) > 1.
  2. expand doesn't change the value of an expression, and an expression without s has the same value at two points that only differ in s.
  3. The fast path finds two such points where f(E_2) - f(E) has different values, and the same for f(E_2) / f(E). It evaluates them exactly, as integers with Floats read as exact rationals. So s has to appear in both expansions. The points only show that it does.
  4. Neither relational can turn into True or False either. Both are false at E_2 = E, and swapping E and E_2 negates the difference and inverts the ratio. If either one were false everywhere, f(E_2) and f(E) would have to be equal at the points it found, and they aren't.

It skips points where f is undefined or not rational (e.g. a pole inside Max), and it only runs when every symbol is positive, so the points are always inside the symbols' domain.

True is also the safe direction. Both callers replace s with 1 on False and keep it on True, so the worst case is less pruning.

I want to be upfront about one case where it isn't identical to main: Float coefficients. Main expands them as floats, so a term in s can round to exactly zero and s gets dropped, where the fast path keeps it. If you'd rather it match main exactly, I can skip the fast path when f has a Float in it. I can also use a fixed list of points in place of the seeded random ones, and put this argument in the docstring.

Is a check like this OK with you? If you'd rather not have any numeric evaluation in the mapper, I understand. I can close this and look for a symbolic way to speed it up.

@tanner-andrulis

Copy link
Copy Markdown
Contributor

OK, that idea sounds good in principle. The current code and docstring is not descriptive of the meaning however. PRs should consist of good human-written code and docstrings that clearly explain what the code is doing.

@prawalpokharel

Copy link
Copy Markdown
Contributor Author

Thanks, and sorry about the docstrings. You're right that they didn't explain what the code does.

To be upfront about how this was written: I used Claude Code for the first version and again for this revision. I went through the argument and the code myself and checked both, and I can explain every line.

What changed in the new commit:

  • _provably_changes_with is now _has_witness_that_s_affects_comparison. Its docstring says what True and False mean, and gives the argument from my last comment for why a True holds for every input.
  • The seeded random values are gone. It now tries a fixed list of primes, and the comments say why any values work and how they're picked. s gets one small and one large value, so a term in s that only wins a Max or Min at one end still shows up. That also makes it settle more calls: on the TPU-v4i case I profiled, it now answers 95 of the 102 calls without expanding, up from 76.
  • The variables are named after what they hold (values_e_2, f_e_at_s_1, ...). Comments explain the case where s is enumerated, why the ratio needs positive values, and why _exact_value rejects some results.
  • The tests are named after what they check. Two are new: one for an enumerated s, and one for _exact_value on undefined and irrational values.
  • I also cut wording that didn't explain anything.

I kept the Float behaviour. All 102 formulas in that case have Float coefficients, so skipping the fast path for Floats would remove most of the speedup. I can change it if you'd rather it match main exactly.

How I checked it: in regression cases 1, 3 and 25, every affects_comparison call was also answered by the symbolic check, and none of the 129 fast-path answers disagreed. The pmapping tables are identical to main's. test_affects_comparison.py and test_mapper.py pass on top of current main. I haven't rerun the full timing yet.

Rename _provably_changes_with to _has_witness_that_s_affects_comparison
and document what a True and a False mean and why a witness proves the
result for every input. Replace the seeded random values with a fixed
list of primes, giving s one small and one large value, and name the
variables after what they hold. Name the tests after what they check,
and add tests for an enumerated symbol and for _exact_value.
@prawalpokharel
prawalpokharel force-pushed the perf/affects-comparison-shortcut branch from 42cfa05 to 85d3548 Compare November 19, 2026 23:36
@prawalpokharel

Copy link
Copy Markdown
Contributor Author

Rebased onto current main, including 8acf39e. I switched the new tests to positive-only symbols to match, and re-ran the full check on the result: all 46 regression cases give identical pmapping tables and joined mappings, and none of the 15,735 witness answers disagree with the symbolic check. The description has the updated numbers.

This branch was previously deployed

1 inactive deployment
pypi — 85d35488 Deployed Nov 19, 2026 by prawalpokharel via Publish package to PyPI #385
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants