Skip to content

feat(blob): add seekable descriptor streams - #763

Merged
JingsongLi merged 1 commit into
apache:mainfrom
XiaoHongbo-Hope:codex/blob-descriptor-stream
Aug 31, 2026
Merged

feat(blob): add seekable descriptor streams#763
JingsongLi merged 1 commit into
apache:mainfrom
XiaoHongbo-Hope:codex/blob-descriptor-stream

Conversation

@XiaoHongbo-Hope

Copy link
Copy Markdown
Contributor

Summary

  • extend the standalone BlobReader from feat(blob): add standalone BlobDescriptor reader #762 with open_blob
  • add lazy, incremental range reads and start/current/end seeking
  • share the existing request and in-flight byte limiter across batch and stream reads
  • preserve catalog-managed credential refresh when reusing table FileIO

This PR contains Rust core support only. C and Go bindings remain separate follow-up work from draft #761. Batch descriptor grouping and IO merge behavior are unchanged.

Testing

  • cargo test -p paimon table::blob_resolver::tests --lib (13 passed)
  • cargo test -p paimon test_blob_stream_reuses_refreshing_file_io --lib
  • cargo clippy -p paimon --all-targets -- -D warnings
  • cargo fmt --all -- --check

@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as ready for review August 30, 2026 13:32

@QuakeWang QuakeWang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JingsongLi
JingsongLi merged commit 7215a56 into apache:main Aug 31, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants