Skip to content

toByteArray should return a copy not a reference - #3729

Merged
Fokko merged 3 commits into
apache:masterfrom
dossett:test-dictionary-page-copy-isolation
Aug 31, 2026
Merged

toByteArray should return a copy not a reference#3729
Fokko merged 3 commits into
apache:masterfrom
dossett:test-dictionary-page-copy-isolation

Conversation

@dossett

@dossett dossett commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

#3565 introduced several optimizations, but I think it also introduced an issue that PR seeks to address. Basically BytesInput.copy(BytesInput) has a public contract to return a copy, but a change to BytesInput.toByteArray() led to returning a reference in some cases. In addition to breaking the public contract, this can lead to data corruption.

See also #3717

* Zero-copy override: returns the backing array directly when fully used,
* skipping the base-class BAOS allocation + copy on every decompressor call.
* Returning the mutable array is safe — the base class already exposes a
* mutable {@code BAOS.getBuf()}.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Returning the mutable array is safe is the key. The array was mutable before, but it was a mutable copy of the original array.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we revert the whole toByteArray method? This effectively reverts the part in:

https://github.com/apache/parquet-java/pull/3565/changes#diff-49155017f14a28333fad01fe81e8c7cd353cdc73824360db5d8ba15b7e7dd6daR693-R707

It is also deprecated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@Fokko Fokko left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this issue and removing the override, less is more 👍

@Fokko
Fokko merged commit 05c0768 into apache:master Aug 31, 2026
4 checks passed
Fokko pushed a commit that referenced this pull request Aug 31, 2026
* BytesInput.copy(BytesInput) should return a copy

* make tests clearer

* Remove unsafe BytesInput toByteArray override
@Fokko Fokko changed the title BytesInput.copy(BytesInput) should return a copy not a reference toByteArray should return a copy not a reference Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants