Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove physical expr of ListIndex and ListRange, convert to array_element and array_slice functions #9492

Merged
merged 2 commits into from
Mar 10, 2024

Conversation

jayzhan211
Copy link
Contributor

Which issue does this PR close?

Closes #.

Preparation of #9285

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
@github-actions github-actions bot added physical-expr Physical Expressions optimizer Optimizer rules core Core DataFusion crate labels Mar 7, 2024
@jayzhan211 jayzhan211 marked this pull request as ready for review March 7, 2024 14:17
@alamb alamb changed the title Remove physical expr of ListIndex and ListRange, move the conversion to optimization Remove physical expr of ListIndex and ListRange, convert to array_element and array_slice functions Mar 10, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really nice PR @jayzhan211 -- thank you so much.

Rewriting the physical expr to use a function is very clever and a very nice reduction in duplication

Screenshot 2024-03-10 at 6 35 12 AM

@@ -119,6 +121,39 @@ impl TreeNodeRewriter for OperatorToFunctionRewriter {
})));
}
}

if let Expr::GetIndexedField(GetIndexedField {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite clever 👏

@@ -42,61 +40,25 @@ use std::{any::Any, sync::Arc};
pub enum GetFieldAccessExpr {
/// Named field, For example `struct["name"]`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 maybe we should also convert this into a function call as well (as a follow on PR) to remove the entire GetFieldAccessExpr physical expr

@jayzhan211
Copy link
Contributor Author

I will file an issue to track the NamedStructField. Thanks @alamb

@jayzhan211 jayzhan211 merged commit 96664ce into apache:main Mar 10, 2024
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate optimizer Optimizer rules physical-expr Physical Expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants