Skip to content

[FLINK-40545][table] Add parseJson() and tryParseJson() Table API expression methods - #29091

Merged
snuyanzin merged 3 commits into
apache:masterfrom
raminqaf:FLINK-40545-json-parse-table-api
Sep 6, 2026
Merged

[FLINK-40545][table] Add parseJson() and tryParseJson() Table API expression methods#29091
snuyanzin merged 3 commits into
apache:masterfrom
raminqaf:FLINK-40545-json-parse-table-api

Conversation

@raminqaf

@raminqaf raminqaf commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

PARSE_JSON and TRY_PARSE_JSON were only reachable from the Table API through genericcall("PARSE_JSON", ...) invocations, unlike the other JSON functions (jsonQuote(), isJson(), jsonValue(), jsonQuery(), ...), which all expose dedicated fluent methods. This pull request closes that gap by adding parseJson() and tryParseJson() to. BaseExpressions, following the existing jsonQuote() pattern, so VARIANT values can be produced with $("f").parseJson() instead of the generic call() syntax.

Brief change log

  • Added parseJson() and tryParseJson() to BaseExpressions (Scala's fluent API inherits these automatically since it extends BaseExpressions)
  • Added the matching parse_json() / try_parse_json() methods to the PyFlink Table API
  • Documented both as Table API equivalents (STRING.parseJson() / STRING.tryParseJson()) in the JSON function reference (docs/data/sql_functions.yml and the zh variant)
  • Updated JsonFunctionsITCase and CastFunctionITCase to use the new fluent methods in place of call("PARSE_JSON", ...) / call("TRY_PARSE_JSON", ...), including two cases hat previously stayed SQL-only for lack of a Table API equivalent

Verifying this change

This change added tests and can be verified as follows:

  • Extended JsonFunctionsITCase with Table API testResult(...) cases for parseJson() and tryParseJson(), paired with their SQL equivalents
  • Converted existing PARSE_JSON/TRY_PARSE_JSON cast tests in CastFunctionITCase to go through the new fluent methods

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yes
    (BaseExpressions is @PublicEvolving; two new methods added, nothing changed or removed)
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components),
    Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? docs / JavaDocs

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Opus 4.8

@flinkbot

flinkbot commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@spuru9 spuru9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions Bot added the community-reviewed PR has been reviewed by the community. label Sep 4, 2026
Comment thread docs/data/sql_functions.yml Outdated
@raminqaf
raminqaf force-pushed the FLINK-40545-json-parse-table-api branch from 0c5f5e1 to fd8ad0d Compare September 4, 2026 11:03

@twalthr twalthr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread docs/data/sql_functions_zh.yml Outdated

variant:
- sql: PARSE_JSON(json_string[, allow_duplicate_keys])
table: STRING.parseJson([BOOLEAN allowDuplicateKeys])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
table: STRING.parseJson([BOOLEAN allowDuplicateKeys])
table: STRING.parseJson([allowDuplicateKeys])

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like not resolved yet
there is still a couple of places in doc with same issue, e.g.
https://github.com/apache/flink/pull/29091/changes#diff-539fb22ee6aeee4cf07230bb4155500c6680c4cc889260e2c58bfa9d63fb7de5R1353

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@snuyanzin Addressed

…ression methods

PARSE_JSON and TRY_PARSE_JSON were only reachable in the Table API through generic call() invocations, unlike other JSON functions (jsonQuote(), isJson(), jsonValue(), etc.) which expose dedicated fluent methods. Add parseJson() and tryParseJson() to BaseExpressions following the existing jsonQuote() pattern, including overloads that accept the allowDuplicateKeys flag supported by the underlying SQL functions. Update the PyFlink Table API accordingly, and document both as Table API equivalents in the JSON function reference.
Existing tests that relied on call("PARSE_JSON", ...) or stayed SQL-only for lack of a Table API equivalent are updated to use the new fluent methods.
@raminqaf
raminqaf force-pushed the FLINK-40545-json-parse-table-api branch from 32e0705 to 1fb7119 Compare September 4, 2026 13:17
@snuyanzin
snuyanzin merged commit bf4a71a into apache:master Sep 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants