Skip to content

馃悰 Spread block hashes to avoid pathological set/dict collisions - #608

Merged
MiWeiss merged 3 commits into
mainfrom
fix/block-hash
Sep 2, 2026
Merged

馃悰 Spread block hashes to avoid pathological set/dict collisions#608
MiWeiss merged 3 commits into
mainfrom
fix/block-hash

Conversation

@MiWeiss

@MiWeiss MiWeiss commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Block.__hash__ uses (type, start_line, raw), both None for blocks built in code, so every such block of a type shares one hash. set() then falls back to __eq__, which compares the full __dict__.

set() over N distinct entries:

N before after
500 0.033 s 0.0001 s
2000 0.375 s 0.0004 s

Fix: mix in discriminators already compared by __eq__, namely entry type and key, string key, preamble and comment text. Values, fields and metadata stay out, as they can be unhashable after middleware. Error-path blocks stay on the base hash; their __eq__ includes an Exception, so instances are never equal anyway.

Tests: 5 cases, including the equal-implies-equal-hash invariant and unhashable values. Suite 2581 passed, from 2576.


馃 Generated with Claude Code

@MiWeiss
MiWeiss force-pushed the fix/block-hash branch 3 times, most recently from 8f5851d to 620a443 Compare September 2, 2026 20:01
`Block.__hash__` only mixed in type, start_line and raw. Blocks created
programmatically have neither a start_line nor a raw value, so every such
block of a given type hashed identically. Building a set or dict of N
constructed blocks thus degenerated into N^2 `__eq__` calls, each of which
compares the full `__dict__` (walking every `Field` of an `Entry`).

`Entry`, `String`, `Preamble`, `ExplicitComment` and `ImplicitComment` now
mix their cheap, hashable identifiers (entry type/key, key, text) into the
hash. Values, fields and parser metadata stay excluded, as they may be
mutable, expensive to hash or outright unhashable after middleware ran.

Deduplicating 2000 constructed entries: 0.375s -> 0.0004s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@MiWeiss

MiWeiss commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

lgtm

@MiWeiss
MiWeiss merged commit 541bc37 into main Sep 2, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant