[Feature] Improve version compatibility tests between old and new Paimon versions - #9504
Conversation
JingsongLi
left a comment
There was a problem hiding this comment.
Requirement fit: PIVOT
Implementation: FINDINGS
[P2] Exercise real old-version readers and files
Location: SnapshotVersionCompatibilityTest#testNewSnapshotJsonCompatibleWithOldReader and SchemaVersionCompatibilityTest#testNewSchemaJsonCompatibleWithOldReader
Problem: Both “old reader” tests serialize with the current classes and immediately deserialize with the same current Snapshot.fromJson / JsonSerdeUtil.fromJson. No old Paimon artifact or old reader is loaded. Likewise, the backward tests use hand-written snapshot/schema JSON only; they do not read metadata and data files produced by an old release.
Trigger: A current change emits JSON, manifest, or data-file content that the current reader accepts but a supported prior Paimon release cannot read. These tests still pass because only the current reader is exercised.
Impact: This PR can go green while neither direction requested by #3531 is actually compatible, giving future format changes a false compatibility gate.
Fix: Run a real cross-version matrix in isolated classloaders/processes: have a pinned prior release write a table (including manifests/data files) and read it with HEAD, then have HEAD write a table and read it with that prior release. Checked-in release-generated fixtures can cover the old-writer/current-reader direction, but the current-writer/old-reader direction still needs the old reader artifact.
Claim: #3531 asks for both new→old and old→new metadata/file compatibility.
Evidence: Every parse in these new tests resolves to the current checkout, and no historical dependency, process, fixture archive, manifest, or data file is used.
Concern: The implementation verifies current-version round trips and permissive current parsing, not cross-version compatibility.
Cost: 839 lines of tests add maintenance while leaving the requested regression boundary unprotected.
Pivot: Replace the synthetic same-version cases with a small, version-pinned end-to-end compatibility harness covering both directions.
Fixes #3531: Add version compatibility tests to verify:
Changes
SnapshotVersionCompatibilityTestfor snapshot-level compatibilitySchemaVersionCompatibilityTestfor schema-level compatibility