fix: invalid sqls when unparsing derived table with columns contains calculations, limit/order/distinct #11756
+139
−69
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rder/distinct (#24) * compare format output to make sure the two level of projects match * add method to find inner projection that could be nested under limit/order/distinct * use format! for matching in unparser sort optimization too * refactor * use to_string and also put comments in
alamb
approved these changes
Aug 5, 2024
| @@ -373,6 +373,38 @@ fn roundtrip_statement_with_dialect() -> Result<()> { | |||
| parser_dialect: Box::new(GenericDialect {}), | |||
| unparser_dialect: Box::new(UnparserDefaultDialect {}), | |||
| }, | |||
| // Test query that has calculation in derived table with columns | |||
There was a problem hiding this comment.
These look like improvements to me
|
Thanks again @y-f-u -- sorry for the delay in merging |
phillipleblanc
deleted the
unparser-fix-for-derived-table-with-column-contains-math-and-limit
branch
phillipleblanc
pushed a commit
to spiceai/datafusion
that referenced
this pull request
Aug 14, 2024
…calculations, limit/order/distinct (apache#11756) * Fix unparser derived table with columns include calculations, limit/order/distinct (#24) * compare format output to make sure the two level of projects match * add method to find inner projection that could be nested under limit/order/distinct * use format! for matching in unparser sort optimization too * refactor * use to_string and also put comments in * clippy * fix unparser derived table contains cast (#25) * fix unparser derived table contains cast * remove dbg
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
This further addressed the unparser issues when derived table contains match operator,
cast,limit,order, etc.Rationale for this change
These are common SQL patterns which doesn't work now. See test cases for details
What changes are included in this PR?
Rewrite more bits of logical plan during unparsing. The key change is to use field name to match the outer layer projection field to detect if the logical plan can be corrected for unparsing purpose, which is the reverse path of parsing for these sql during the first leg of the round trip.
Are these changes tested?
Yes
Are there any user-facing changes?
No