Skip to content

test: implement TrickleReader as a stream, not a readVec - #46

Merged
lalinsky merged 1 commit into
mainfrom
fix/trickle-reader-stream
Sep 5, 2026
Merged

test: implement TrickleReader as a stream, not a readVec#46
lalinsky merged 1 commit into
mainfrom
fix/trickle-reader-stream

Conversation

@lalinsky

@lalinsky lalinsky commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Fixes #44. Replaces #45, which fixed one symptom while leaving the cause in place.

stream is the only operation a std.Io.Reader must supply — discard, readVec and rebase all default to implementations derived from it:

stream:  *const fn (...)                      // no default
discard: *const fn (...) = defaultDiscard
readVec: *const fn (...) = defaultReadVec
rebase:  *const fn (...) = defaultRebase

TrickleReader had that inverted. It stubbed stream out to return error.EndOfStream and hand-wrote readVec and discard instead, so it was correct only in the situations it happened to be tried in. Every problem with it followed from that:

  • readVec discarded the destination it was handed and returned 1 anyway. Under readSliceShort that both lost the byte and miscounted it; once the reader's own buffer filled it returned 0 with no progress and the caller spun forever.
  • discard returned EndOfStream whenever a skip reached past what was already buffered, so skipAny could not run against it.
  • stream would have broken anything reaching for streamRemaining or readAlloc.

Implementing stream and deleting the other two overrides is less code and correct everywhere. defaultReadVec already does the empty-vs-non-empty destination handling, and better — it picks whichever of data[0] or r.buffer is larger. defaultDiscard routes through a Writer.Discarding.

fn stream(r: *std.Io.Reader, w: *std.Io.Writer, limit: std.Io.Limit) std.Io.Reader.StreamError!usize {
    const self: *TrickleReader = @fieldParentPtr("reader", r);
    if (self.pos >= self.data.len) return error.EndOfStream;
    if (!limit.nonzero()) return 0;

    try w.writeByte(self.data[self.pos]);
    self.pos += 1;
    return 1;
}

Net −19 lines from the type, +2 tests.

Two tests this unblocks

Neither could run before:

  • A 200-byte string through buffers of 16, 33 and 64 bytes. String values are copied out rather than borrowed, so unlike map keys they have no size limit relative to the reader's buffer. This used to hang; I had to kill the run at 45s.
  • skipAny over a value larger than the buffer, checking it lands exactly on the value that follows. This used to fail with error.EndOfStream despite the bytes being available — which matters now that skip_unknown_fields is built on skipAny.

Why #45 was the wrong fix

It made readVec handle both drivers correctly, which removed the hang, but left discard and stream stubbed. The next gap would have been discard the moment someone tested skip_unknown_fields against a streaming reader — and the one after that, stream. Deriving everything from one correct primitive removes the whole class instead of one instance.

zig build test: 185/185 pass, up from 183.

stream is the only operation a std.Io.Reader must supply. discard, readVec and
rebase all default to implementations derived from it. TrickleReader had that
inverted: it stubbed stream out to return EndOfStream and hand-wrote readVec
and discard instead, so it was correct only in the situations it happened to be
tried in.

Every problem with it followed from that. readVec discarded the destination it
was handed and returned 1 anyway, which under readSliceShort both lost the byte
and miscounted it; once the reader's own buffer filled it returned 0 with no
progress and the caller spun forever. discard returned EndOfStream whenever a
skip reached past what was already buffered, so skipAny could not run against
it. stream itself would have broken anything reaching for streamRemaining.

Implementing stream and deleting the other two overrides is less code and
correct everywhere. defaultReadVec already picks whichever of the caller's
destination or the reader's buffer is larger, which is better than the
hand-written version, and defaultDiscard routes through a Writer.Discarding.

Two tests this unblocks, neither of which could run before:

  - a 200 byte string decoded through buffers of 16, 33 and 64 bytes. String
    values are copied out rather than borrowed, so unlike map keys they have no
    size limit relative to the reader's buffer. This one used to hang.
  - skipAny over a value larger than the buffer, landing exactly on the value
    that follows. This one used to fail with EndOfStream.
@coderabbitai

coderabbitai Bot commented Sep 5, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 43 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 116f1adc-2e89-4485-9054-16bf6842bd6e

📥 Commits

Reviewing files that changed from the base of the PR and between 52b624d and 5ae50ac.

📒 Files selected for processing (1)
  • src/msgpack.zig

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lalinsky
lalinsky merged commit 2d197d5 into main Sep 5, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TrickleReader hangs on values larger than its buffer, which is what it claims to model

1 participant