Skip to content

fix(mcp): keep grep parsing aligned on record boundaries - #2066

Open
kavish-19 wants to merge 1 commit into
DeusData:mainfrom
kavish-19:fix-grep-record-alignment
Open

fix(mcp): keep grep parsing aligned on record boundaries#2066
kavish-19 wants to merge 1 commit into
DeusData:mainfrom
kavish-19:fix-grep-record-alignment

Conversation

@kavish-19

Copy link
Copy Markdown

Fixes #2011.

Symptom

search_code returns file paths and line numbers that exist in no file. A grep record longer than the 2 KiB read buffer is split across two fgets calls, and the continuation is parsed as a fresh file:line:content record — the text before its first delimiter becomes the file, the text after it becomes the line number. One minified or generated line containing colons is enough to trigger it.

An agent reading that result calls get_code_snippet on a path that was never in the repository.

The same split can also lose a real match: when the tail happens to contain fewer than two delimiters the record is dropped by the continue instead, so the count moves in either direction.

Root cause

collect_grep_matches() in src/mcp/mcp.c:

char line[CBM_SZ_2K];                       /* 2048 */

while (fgets(line, sizeof(line), fp) && gm_count < grep_limit) {
    ...
    char *sep1 = strchr(line, (unsigned char)sep);   if (!sep1) { continue; }
    char *sep2 = strchr(sep1 + SKIP_ONE, (unsigned char)sep); if (!sep2) { continue; }

Nothing distinguishes a continuation chunk from a record start.

The fix

fgets already reports the truncation: a filled buffer with no trailing newline is a partial record. Drain the remainder of that record before parsing.

Placement matters — the drain sits before the early continue paths (empty line, missing delimiter, path_filter rejection), so every exit from an over-long record leaves the stream aligned on a record boundary. Draining after them would leave the tail to be read as a record by the next iteration on exactly the paths that already discard data.

The head chunk is still emitted as the real match it is. Its content was already bounded by the 1 KiB content field, well under the 2 KiB read buffer, so no reported text changes.

Verification

Running the parsing loop against the issue's own input — one 2936-byte record plus one normal record, in a repository with two matches:

BEFORE (main)
  match 1: file=big.js                   line=1
  match 2: file=                         line=0     <- invented; not a file
  match 3: file=normal.js                line=1
  total_grep_matches = 3

AFTER
  match 1: file=big.js                   line=1
  match 2: file=normal.js                line=1
  total_grep_matches = 2

That reproduces the issue exactly: a fabricated row carrying line: 0, and total_grep_matches: 3 for a repository containing 2 matches.

To be precise about what that run is: it is the collect_grep_matches read/parse loop compiled standalone, with and without the new branch, over the same grep output — evidence about the loop, not an end-to-end run of the built binary. See the next section for why I could not run the real suite here.

An end-to-end regression test is added to tests/test_mcp.c as search_code_long_line_does_not_invent_matches, next to the existing search_code_path_filter_* tests and registered in the suite. It writes a >2 KiB colon-laden line plus a normal one, calls search_code through cbm_mcp_server_handle, and asserts total_grep_matches == 2 and that no row carries line: 0.

What I could not run locally

Same environment limitation I noted on #2065, stated rather than glossed:

  • scripts/build.sh fails at internal/cbm/preprocessor.cpp with fatal error: 'cctype' file not found — the C++ standard-library headers are missing from this machine's Command Line Tools. It is a local toolchain fault, unrelated to this change, and it blocks the test runner because that links preprocessor.o too.
  • clang-format, clang-tidy and cppcheck are not installed here.

What I did instead: src/mcp/mcp.c compiles clean under the project's own warning set (-std=c11 -Wall -Wextra -Werror -Wno-unused-parameter -Wno-sign-compare), and my added test region compiles clean under the same flags. (tests/test_mcp.c shows 3 unrelated errors at line 1597 under my ad-hoc include set, identically on a clean main checkout — a generated header my flags do not supply, not something this branch introduces.) CI is the authority on the full suite and the linters; if it flags anything I will fix it promptly.

Not included

Validating the parsed line number (rejecting non-numeric or <= 0) is listed in the issue as a second line of defence. Record alignment is the root cause and removes the fabricated rows at the source, so I kept this PR to that one cure per CONTRIBUTING's one-issue-per-PR rule. Happy to send the validation separately if you want it.

search_code reports file paths and line numbers that exist in no file. A
grep record longer than the 2 KiB read buffer comes back from fgets split
across two calls, and collect_grep_matches parses the continuation as a
fresh file:line:content record: the text before its first delimiter
becomes the file, the text after it becomes the line number. A single
minified or generated line containing colons is enough. An agent then
calls get_code_snippet on a path that was never in the repository.

The same split can also lose a real match: when the tail happens to hold
fewer than two delimiters the record is skipped by the `continue` above
instead, so the count moves in either direction.

fgets already reports this — a filled buffer with no trailing newline is
a partial record. Drain the remainder of that record before parsing, and
do it before the early `continue` paths so every exit leaves the stream
on a record boundary. The head chunk is still emitted as the real match
it is; its content was already bounded by the 1 KiB content field, so no
reported text changes.

Closes DeusData#2011

Signed-off-by: kavish-19 <sworks.dev@gmail.com>
@kavish-19
kavish-19 requested a review from DeusData as a code owner September 5, 2026 07:13
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

  • It will not be closed for inactivity. No stale bot touches pull requests here.
  • It may still sit a while before a human reads it. That is on us, not on you.
  • Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

  • Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
  • Get CI green, or say which failures you believe are pre-existing.
  • Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
  • Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

search_code invents file paths and line numbers: fgets splits a >2047-byte grep record and the continuation is parsed as a fresh file:line:content

2 participants