fix(mcp): keep grep parsing aligned on record boundaries - #2066
Open
kavish-19 wants to merge 1 commit into
Open
Conversation
search_code reports file paths and line numbers that exist in no file. A grep record longer than the 2 KiB read buffer comes back from fgets split across two calls, and collect_grep_matches parses the continuation as a fresh file:line:content record: the text before its first delimiter becomes the file, the text after it becomes the line number. A single minified or generated line containing colons is enough. An agent then calls get_code_snippet on a path that was never in the repository. The same split can also lose a real match: when the tail happens to hold fewer than two delimiters the record is skipped by the `continue` above instead, so the count moves in either direction. fgets already reports this — a filled buffer with no trailing newline is a partial record. Drain the remainder of that record before parsing, and do it before the early `continue` paths so every exit leaves the stream on a record boundary. The head chunk is still emitted as the real match it is; its content was already bounded by the 1 KiB content field, so no reported text changes. Closes DeusData#2011 Signed-off-by: kavish-19 <sworks.dev@gmail.com>
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2011.
Symptom
search_codereturns file paths and line numbers that exist in no file. A grep record longer than the 2 KiB read buffer is split across twofgetscalls, and the continuation is parsed as a freshfile:line:contentrecord — the text before its first delimiter becomes the file, the text after it becomes the line number. One minified or generated line containing colons is enough to trigger it.An agent reading that result calls
get_code_snippeton a path that was never in the repository.The same split can also lose a real match: when the tail happens to contain fewer than two delimiters the record is dropped by the
continueinstead, so the count moves in either direction.Root cause
collect_grep_matches()insrc/mcp/mcp.c:Nothing distinguishes a continuation chunk from a record start.
The fix
fgetsalready reports the truncation: a filled buffer with no trailing newline is a partial record. Drain the remainder of that record before parsing.Placement matters — the drain sits before the early
continuepaths (empty line, missing delimiter,path_filterrejection), so every exit from an over-long record leaves the stream aligned on a record boundary. Draining after them would leave the tail to be read as a record by the next iteration on exactly the paths that already discard data.The head chunk is still emitted as the real match it is. Its content was already bounded by the 1 KiB
contentfield, well under the 2 KiB read buffer, so no reported text changes.Verification
Running the parsing loop against the issue's own input — one 2936-byte record plus one normal record, in a repository with two matches:
That reproduces the issue exactly: a fabricated row carrying
line: 0, andtotal_grep_matches: 3for a repository containing 2 matches.To be precise about what that run is: it is the
collect_grep_matchesread/parse loop compiled standalone, with and without the new branch, over the same grep output — evidence about the loop, not an end-to-end run of the built binary. See the next section for why I could not run the real suite here.An end-to-end regression test is added to
tests/test_mcp.cassearch_code_long_line_does_not_invent_matches, next to the existingsearch_code_path_filter_*tests and registered in the suite. It writes a >2 KiB colon-laden line plus a normal one, callssearch_codethroughcbm_mcp_server_handle, and assertstotal_grep_matches == 2and that no row carriesline: 0.What I could not run locally
Same environment limitation I noted on #2065, stated rather than glossed:
scripts/build.shfails atinternal/cbm/preprocessor.cppwithfatal error: 'cctype' file not found— the C++ standard-library headers are missing from this machine's Command Line Tools. It is a local toolchain fault, unrelated to this change, and it blocks the test runner because that linkspreprocessor.otoo.clang-format,clang-tidyandcppcheckare not installed here.What I did instead:
src/mcp/mcp.ccompiles clean under the project's own warning set (-std=c11 -Wall -Wextra -Werror -Wno-unused-parameter -Wno-sign-compare), and my added test region compiles clean under the same flags. (tests/test_mcp.cshows 3 unrelated errors at line 1597 under my ad-hoc include set, identically on a cleanmaincheckout — a generated header my flags do not supply, not something this branch introduces.) CI is the authority on the full suite and the linters; if it flags anything I will fix it promptly.Not included
Validating the parsed line number (rejecting non-numeric or
<= 0) is listed in the issue as a second line of defence. Record alignment is the root cause and removes the fabricated rows at the source, so I kept this PR to that one cure per CONTRIBUTING's one-issue-per-PR rule. Happy to send the validation separately if you want it.