index_repository: one unreadable directory aborts the whole run with a generic `Pipeline failed`, and the hint points elsewhere (0.10.8)

[balerdis]

Summary

One directory the indexing user cannot read (chmod 000) aborts the whole run with a generic Pipeline failed. Nothing else is indexed, no partial result is kept, and the hint sends the reader to look for the wrong thing:

Pipeline failed. Check repo_path exists and contains source files.
Try mode='fast' for a quicker diagnostic run.

The path exists and does contain source files. mode='fast' fails the same way.

Directories excluded by .gitignore, .cbmignore or the built-in skip list are walked past without incident. An unreadable one is the same situation from the walker's point of view — a directory whose contents will not be indexed — but it takes the run down.

Minimal reproduction

mkdir -p repro/a repro/b repro/locked && cd repro && git init -q .
printf '<?php\nfunction alpha() { return 1; }\n' > a/alpha.php
printf '<?php\nfunction beta() { return alpha(); }\n' > b/beta.php
printf 'x\n' > locked/secret.txt
chmod 000 locked
git add -A && git commit -qm init

Then index_repository(repo_path=".../repro", mode="full").

Observed

{"project": "...", "status": "error",
 "hint": "Pipeline failed. Check repo_path exists and contains source files. Try mode='fast' for a quicker diagnostic run."}

Flip the single bit back — same tree, same files, nothing else touched:

chmod 755 locked

{"nodes": 10, "edges": 10, "skipped_count": 0, "status": "indexed"}

Expected

An unreadable directory should be reported and skipped, the way an excluded one is — ideally in not_indexed_files (or skipped) with reason: "permission denied" — and the rest of the repository should still be indexed. A permission problem in one corner is not a reason to have no graph at all.

If aborting is deliberate, the error should at least name the offending path, since the current text actively points away from the cause.

Why this is easy to hit

It showed up on a PHP application served by Apache, where runtime/.../images/ belongs to www-data with mode 0770 and the developer's own account is not in that group. That is an ordinary deployment layout, and the directory holds uploaded files — nothing anyone wants indexed anyway. The repository could not be indexed at all until the directory was named in .cbmignore, which requires already knowing the cause.

The misdiagnosis cost is real: the failure was attributed to nested git checkouts for a long time (this repository has four). Excluding those changed nothing; excluding the unreadable directory fixed it immediately, and the nested checkouts index fine.

Notes

• Version: 0.10.8, Linux x86-64, stdio server.
• Same behaviour in mode='full' and mode='fast'.

[AmirF194]

Reproduced on HEAD 5802ccd with your exact repro, same output.

One correction: the abort isn't coming from the opendir failure itself. I instrumented the opendir branch in walk_dir and it does fire, but discovery finishes clean right after: files=2, elapsed_ms=20, no error logged. The pipeline dies silently a step later, somewhere between the semantic manifest hash pass and the next checkpoint, nothing logged in between. Same shape as the PERSIST_FAILED silence #1620 fixed, just further down the pipeline. Haven't pinned the exact line yet.

[DeusData]

Thank you for the minimal reproduction, and thank you @AmirF194 for checking it on current main.

The follow-up usefully narrows this: the unreadable-directory opendir failure is observed, but discovery still completes with the two readable files, so the terminal failure is later in the pipeline. Separately, the MCP error branch still collapses every nonzero pipeline result into the same repo_path hint, so the user-facing diagnosis is definitely wrong even before the later failure is pinned down.

I have routed this as a high-priority indexing and diagnostics bug. We need more time to isolate the exact post-discovery failure path; no further reporter action is requested right now. Thank you both for separating the observed cause from the first plausible hypothesis.