⚡ Fix quadratic Library.add during parsing and slow quote-enclosing check - #612
Merged
Conversation
`_find_duplicate_keys` copied all existing entry and string keys into sets on every call, making `add()` O(library size) per call and parsing quadratic in the number of comment/preamble/failed blocks (the splitter added those with the default `fail_on_duplicate_key=True`). Look keys up in the by-key dicts directly and pass `fail_on_duplicate_key=False` for all blocks added by the splitter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`_is_enclosed_in_quotes` (added in #604) scanned every quoted value with a regex, making the default parse stack ~4x slower on files with quoted values. Skip the scan when the value contains no inner quote, and avoid the slice copy in the corresponding brace fast path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MiWeiss
force-pushed
the
perf/fix-quadratic-add-and-enclosing
branch
from
September 3, 2026 19:39
8a612c1 to
80792c1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two speed regressions on
mainvsv2.0.0b9, found by benchmarking on cryptobib, the ACL anthology and generated files.1. Parsing is quadratic in the number of comment blocks (since #527).
Library.addnow defaults tofail_on_duplicate_key=True, and_find_duplicate_keyscopied both by-key dicts into sets on every call. The splitter passedFalseonly for entries and strings, so each comment, preamble or failed block cost O(entries so far).Fix: look keys up in the dicts directly (cost depends only on the blocks being added) and pass
Falseat all splitter call sites. Regression test added.2.
RemoveEnclosingMiddlewareslow on quote-enclosed values (since #604)._is_enclosed_in_quotesran a regex over every value; the brace variant had a fast path, the quote one didn't.Fix: return early when the value has no inner
"(the scan can only return False on one).Min of 3, seconds:
%line before each —split()split()parse_stringparse_stringLibrary.add()callsTests: 2720 passed.
🤖 Generated with Claude Code