Skip to content

[iceberg] Refuse types the mirror cannot publish before the commit or the replace - #9477

Draft
plusplusjiajia wants to merge 2 commits into
apache:masterfrom
plusplusjiajia:gh-iceberg-veto-unsupported-types
Draft

[iceberg] Refuse types the mirror cannot publish before the commit or the replace#9477
plusplusjiajia wants to merge 2 commits into
apache:masterfrom
plusplusjiajia:gh-iceberg-veto-unsupported-types

Conversation

@plusplusjiajia

@plusplusjiajia plusplusjiajia commented Aug 30, 2026

Copy link
Copy Markdown
Member

Purpose

Blob and vector columns have no Iceberg type, and variant needs format version 3, but the mirror only finds out inside IcebergDataField while converting a schema it has already been told to publish, so the snapshot is durable by then and every later commit keeps failing the same way; this validates the same rule where the timestamp and time rules already run, on the current schema at DDL time and on the whole schema history when the mirror is enabled on an existing table, so the refusal lands before the commit instead of after it, and the conversion-layer checks stay as backstops for tables created before this guard.
replaceTable drops or truncates the table it replaces before the new schema reaches the validation in SchemaManager, and neither is undone when that validation refuses the schema, so a replacement carrying one of these types took the old data with it; this judges the replacement schema, directives expanded, before either destructive step, which also covers every other rule validateTableSchema enforces

@plusplusjiajia
plusplusjiajia force-pushed the gh-iceberg-veto-unsupported-types branch from 3313d14 to 0c38c87 Compare August 30, 2026 13:34
@plusplusjiajia
plusplusjiajia marked this pull request as ready for review August 30, 2026 13:35
@plusplusjiajia plusplusjiajia changed the title [iceberg] Refuse types the mirror cannot publish before the commit [iceberg] Refuse types the mirror cannot publish before the commit or the replace Aug 30, 2026
@plusplusjiajia
plusplusjiajia marked this pull request as draft August 30, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant