[FEA] Improve organization of AST code

[vyasr]

Now that conditional joins have been merged, I'll use this PR as a checklist for the various improvements that we'd like to make to the code base:

• Remove dependence of conditional joins on hash joins: Refactor conditional joins #8815
• Separate AST internals from compute_column implementation: Combine linearizer and ast_plan #8900, Move AST evaluator into a separate header #8930
• Hide all AST details from the user, expose only expression construction: Combine linearizer and ast_plan #8900, Move AST evaluator into a separate header #8930
• Move compute_column API to a user-facing namespace outside of ast: Move compute_column API out of ast namespace #8957
• Expose null handling on a per-operator basis rather than a single option for null equality comparisons Make AST operators nullable #9096
• WONTFIX: Upgrade arity dispatching: either infer arity directly from the operator functions, or encode this data at a higher level and remove all the extra functors The fact that operator() is itself templated for the various operator_functor specializations makes this impossible without some very ugly workarounds that hurt readability and are in any case not robust to all possible cases.
• WONTFIX: Improve device_data_reference/expression_result structure. I've experimented with changing device_data_reference from a POD struct into something with methods to encapsulate reading and writing logic and unify the expression_result class, but the amount of divergence between different reference types and output types means that the extra logic involved in dispatching appropriately obviates any real gains, at least with the current architecture.
• Accept more parameters (including expressions) via const ref rather than by value. Expose expression base class publicly and simplify public AST API #9045
• Expose node class publicly so that client code can build up the list of nodes irrespective of type. Expose expression base class publicly and simplify public AST API #9045
• Add virtual destructor for nodes so that all node pointers are destructible. Expose expression base class publicly and simplify public AST API #9045

This issue was originally created in from #7418 (comment), but has since been heavily edited to reflect the updated task list.

[harrism]

@vyasr how did you create an issue without choosing the type? Usually the system auto-populates some labels...

[vyasr]

You can create an issue from a comment on another PR or Issue by clicking on the three dots on the right and selecting "Reference in a new issue" (see the picture below). Issues created this way get the "Originally posted by..." note at the end of the issue. I'm surprised that it bypasses the auto-labeling though...

[vyasr]

Now that conditional joins have been merged, I'll use this issue as a checklist for the various improvements that we'd like to make to the code base: the checklist is now in the issue description.

[harrism]

I'll use this PR as a checklist

This is an issue. :) But in general checklists should go into the opening comment (the issue description). This way the checklist counter shows up in github's other interfaces (e.g. project boards).

[vyasr]

All good points :) I'll move the checklist.