perf(mcp): filter suffix globs while building search scope

[ertankucukoglu]

Summary

• Filter safe simple suffix globs while the indexed Windows file list is built, before PowerShell starts.
• Keep complex patterns on the existing path and preserve the final post-filter.
• Preserve case-insensitive Windows suffix matching.

Performance

Benchmark host: Windows 11 Pro, AMD Ryzen 9 7950X, 64 GB RAM.

Dataset: a mixed-language Windows index with roughly 184,000 indexed files; the *.pas search selected about 460 files. Median elapsed time across three runs improved from 2.10s to 0.63s (about 70%).

Tests

• make -f Makefile.cbm test-focused TEST_SUITES=mcp CC=clang CXX=clang++ SANITIZE=
• 243 passed, 9 skipped

Refs #1565

[github-actions]

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

• It will not be closed for inactivity. No stale bot touches pull requests here.
• It may still sit a while before a human reads it. That is on us, not on you.
• Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

• Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
• Get CI green, or say which failures you believe are pre-existing.
• Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
• Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

[DeusData]

Thank you for the measured Windows improvement and for preserving the existing complex-pattern and final-filter paths. The focused tests and before-and-after timing make the intended scope clear. The contribution queue is quite full, so review may take a little time, but we have the PR routed and will return with feedback as soon as possible.

[DeusData]

Approved. A 70% cut on a real Windows index is worth having, and the way you got it is the safe way.

You reused the existing classifier rather than inventing a notion of "safe". cbm_search_code_file_pattern_can_prefilter was already on main at mcp.c:9706, already trusted for the PowerShell -like optimisation, and already strict: the pattern must be *. followed only by [a-zA-Z0-9._-], so anything carrying a real glob metacharacter — *.{c,h}, *.c*, test_*.c — is rejected and falls to the existing path. Extending a predicate that already had to be right is a much smaller risk than adding a second opinion about which patterns are simple.

The pre-filter is a pure optimisation, and that is the property that matters. It excludes exactly what the surviving post-filter would exclude, so a file cannot be dropped that the old path would have kept. Keeping the post-filter rather than trusting the new one to be complete is the right call.

Your negative test is the one that earns confidence. UnitMain.pas.bak against *.pas must be false, and a naive strstr would happily match it. Anchoring at path_len - suffix_len is what makes that work, and asserting it is what stops someone "simplifying" it later. index.D.TS against *.d.ts covers the multi-dot suffix case that a single strrchr('.') implementation would get wrong.

One nicety, not a change request: the case folding is ASCII-only, which is correct here because can_prefilter already constrains the pattern to ASCII by construction — so there is no culture-dependent comparison to disagree with PowerShell's -like. Worth knowing it is safe by construction rather than by luck, given how much Windows non-ASCII path handling this repo has had to fix.

One small ask: the benchmark would be more useful with its context recorded — machine, and roughly what "a large Windows index" was in files or size. "2.10s to 0.63s, median of three" is a good measurement; without the host it is hard to compare against later. Convention here is to note the machine alongside the numbers.

Please rebase — this is currently DIRTY because #1703 landed in src/mcp/mcp.c yesterday. main also went briefly broken and was repaired by #1993, so rebase onto current main rather than anything older.

[DeusData]

Merged as 96398031. Thank you for rebasing — and sorry it sat: this was approved some time ago and the only thing standing between it and main was our review queue, not your code.

Before merging I verified the combined tree rather than this PR alone, because #1877 landed in the same batch and also touches src/mcp/mcp.c and tests/test_mcp.c. Built main plus all three of the batch and ran mcp cypher graph_buffer: clean build, 493 passed, 0 failed. A per-PR check would not have exercised that interaction.