Skip to content

search_code invents file paths and line numbers: fgets splits a >2047-byte grep record and the continuation is parsed as a fresh file:line:content #2011

Description

@metehanulusoy

What happens

collect_grep_matches() reads the scan output with fgets into a 2048-byte buffer and parses each line as file:line:content. A grep record longer than 2047 bytes is split across two fgets calls, and the continuation chunk is then parsed as if it were a fresh record: the text before its first : becomes the file, and the text after it becomes the line number.

Any file with a line longer than ~2 KB containing colons reproduces it — a minified bundle, generated code, a long data row.

Reproduction

mkdir -p /tmp/fx && cd /tmp/fx && git init -q
python3 -c "
pairs = ','.join('k%d:\"v%d\"' % (i, i) for i in range(260))
open('big.js','w').write('var CFG=({' + pairs + ',NEEDLEmarker:1});\n')
open('normal.js','w').write('const NEEDLEmarker = 42;\n')
"
git add -A && git -c user.email=t@t -c user.name=t commit -qm init
codebase-memory-mcp cli index_repository --repo-path /tmp/fx
codebase-memory-mcp cli --json search_code --project <project> --pattern NEEDLEmarker

big.js is a single 2927-byte line. The response comes back isError: false, and contains:

raw: 1  (cols: file line content)
  "v181",k182     0     "v183",k184:"v184",k185:"v185", … ,NEEDLEmarker:1});

total_grep_matches: 3
total_results: 2

The file is a fragment of JavaScript — "v181",k182 — and the line number is 0. No such file exists. total_grep_matches is 3 for a repository containing 2 matches, because the split produced a third record.

An agent that reads this calls get_code_snippet on a path that was never in the repository, or reports to the user that a symbol lives at a file and line that do not exist.

Mechanism

src/mcp/mcp.c:10261 collect_grep_matches():

char line[CBM_SZ_2K];                       /* 2048 — src/foundation/constants.h:36 */

while (fgets(line, sizeof(line), fp) && gm_count < grep_limit) {
    ...
    char *sep1 = strchr(line, (unsigned char)sep);
    if (!sep1) { continue; }
    char *sep2 = strchr(sep1 + SKIP_ONE, (unsigned char)sep);
    if (!sep2) { continue; }
    *sep1 = '\0';
    *sep2 = '\0';
    const char *path = line;                                  /* mcp.c:10300 */
    ...
    snprintf(gm[gm_count].file, sizeof(gm[0].file), "%s", file);
    gm[gm_count].line = (int)strtol(sep1 + SKIP_ONE, NULL, CBM_DECIMAL_BASE);

Nothing distinguishes a continuation chunk from a record start. When the tail happens to contain two colons the chunk becomes a fabricated match; when it does not, the continue silently drops a genuine one instead — so the same split can either invent a match or lose one.

Suggested fix

Detect the truncation fgets already tells you about: a chunk that does not end in \n is a partial record. Either accumulate until the newline arrives, or consume and discard the remainder of the over-long record and count it as skipped, rather than feeding the tail back into the file:line:content parser. Validating the parsed line number (reject a non-numeric or <= 0 field) would catch the fabricated rows as a second line of defence, but on its own it would still drop the real match silently.

Environment

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    cypherCypher query language parser/executor bugs

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions