planner/cascades: implement ImplementationRule for Selection #12257

francis0407 · 2019-09-18T08:35:26Z

What problem does this PR solve?

Introduce EngineType in cascades planner.
Implement ImplementationRule for LogicalSelection in cascades planner.
Move some tests of cascades integration tests to json file.

Check List

Tests

Integration test

codecov · 2019-09-18T08:40:17Z

Codecov Report

Merging #12257 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #12257   +/-   ##
===========================================
  Coverage   80.0296%   80.0296%           
===========================================
  Files           465        465           
  Lines        106613     106613           
===========================================
  Hits          85322      85322           
  Misses        14909      14909           
  Partials       6382       6382

planner/cascades/implementation_rules.go

planner/implementation/simple_plans.go

francis0407 · 2019-09-18T13:32:57Z

/rebuild

francis0407 · 2019-09-23T05:45:24Z

/rebuild

sre-bot · 2019-09-25T05:03:13Z

@zz-jason, @eurekaka, @winoros, PTAL.

…lection

…into impl_selection

sre-bot · 2019-09-27T05:02:00Z

@zz-jason, @eurekaka, @winoros, PTAL.

sre-bot · 2019-10-07T08:24:23Z

@zz-jason, @eurekaka, @winoros, @lzmhhh123, PTAL.

…lection # Conflicts: # planner/cascades/integration_test.go

winoros

lgtm

…lection

sre-bot · 2019-10-17T10:47:17Z

@zz-jason, @winoros, @lamxTyler, @eurekaka, @lzmhhh123, PTAL.

francis0407 · 2019-10-17T12:33:52Z

/run-unit-test

francis0407 · 2019-10-17T12:43:08Z

/run-unit-test

eurekaka · 2019-10-17T13:39:30Z

planner/cascades/implementation_rules.go

+	logicalSel := expr.ExprNode.(*plannercore.LogicalSelection)
+	physicalSel := plannercore.PhysicalSelection{
+		Conditions: logicalSel.Conditions,
+	}.Init(logicalSel.SCtx(), expr.Group.Prop.Stats.ScaleByExpectCnt(reqProp.ExpectedCnt), logicalSel.SelectBlockOffset(), reqProp)


Shall we make a copy of the reqProp to pass it as children properties? see #8635 for explanations.

Sure. I just find another problem.
The Stats for init should be expr.Group.Prop.Stats instead of expr.Group.Prop.Stats.ScaleByExpectCnt(reqProp.ExpectedCnt)?

And reqProp.ExpectedCnt also should be scaled by the selectivity?

eurekaka · 2019-10-17T14:14:20Z

planner/implementation/simple_plans.go

+
+// CalcCost implements Implementation CalcCost interface.
+func (sel *TiDBSelectionImpl) CalcCost(outCount float64, childCosts []float64, children ...*memo.Group) float64 {
+	sel.cost = outCount*sel.plan.SCtx().GetSessionVars().CPUFactor + childCosts[0]


We should use row count of child group here instead of outCount?

Oh, it seems we didn't update the stats of the child group. If we scale the stats of child group by ExpectedCnt, we can use the row count of the child group then.

eurekaka · 2019-10-17T14:15:02Z

planner/implementation/simple_plans.go

+
+// CalcCost implements Implementation CalcCost interface.
+func (sel *TiKVSelectionImpl) CalcCost(outCount float64, childCosts []float64, children ...*memo.Group) float64 {
+	sel.cost = outCount*sel.plan.SCtx().GetSessionVars().CopCPUFactor + childCosts[0]


planner/memo/expr_iterator.go

planner/memo/pattern.go

sre-bot · 2019-10-20T06:37:17Z

@zz-jason, @winoros, @lamxTyler, @eurekaka, @lzmhhh123, PTAL.

eurekaka

LGTM

sre-bot · 2019-10-21T08:20:21Z

/run-all-tests

sre-bot · 2019-10-21T08:24:32Z

/run-all-tests

…#12257)

implement ImplementationRule for Selection

00b8a33

francis0407 added the sig/planner SIG: Planner label Sep 18, 2019

francis0407 requested review from zz-jason, eurekaka and winoros September 18, 2019 08:35

zz-jason reviewed Sep 18, 2019

View reviewed changes

planner/cascades/implementation_rules.go Show resolved Hide resolved

francis0407 added 2 commits September 18, 2019 17:02

address comment

8d261fb

revert change

b32365d

francis0407 commented Sep 18, 2019

View reviewed changes

planner/implementation/simple_plans.go Outdated Show resolved Hide resolved

francis0407 added 2 commits September 23, 2019 12:02

Split SelectionImpl into RootSelectionImpl and TiKVCopSelectionImpl.

4f61d6c

add comments for EngineTypes

151182a

francis0407 added the type/new-feature label Sep 23, 2019

francis0407 requested a review from zz-jason September 23, 2019 04:24

francis0407 added the status/PTAL label Sep 23, 2019

add test case for root selection

623a8bb

francis0407 added 6 commits September 26, 2019 10:12

Merge branch 'master' of https://github.com/pingcap/tidb into impl_se…

90c3f43

…lection

Merge branch 'master' of https://github.com/pingcap/tidb into impl_se…

9d1cf40

…lection

add engine type in pattern

df118b1

Merge remote-tracking branch 'origin/impl_selection' into impl_selection

5b2f2d4

Merge branch 'impl_selection' of https://github.com/francis0407/tidb …

95374f2

…into impl_selection

fix comment

0e0ffda

francis0407 requested a review from lzmhhh123 September 27, 2019 05:11

francis0407 mentioned this pull request Sep 27, 2019

planner/cascades: introduce EngineType in physical property (experimental) #12437

Closed

fix implementation name

768dd21

francis0407 added 3 commits October 12, 2019 10:22

Merge branch 'master' of https://github.com/pingcap/tidb into impl_se…

9c5bf85

…lection # Conflicts: # planner/cascades/integration_test.go

resolve conflicts

f3d2def

Merge remote-tracking branch 'origin/impl_selection' into impl_selection

78aca89

francis0407 requested review from zz-jason and alivxxx October 15, 2019 03:16

winoros reviewed Oct 15, 2019

View reviewed changes

francis0407 added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 15, 2019

francis0407 added 3 commits October 15, 2019 18:43

Add a test case for the pattern without a leaf AnyOperand

4e52cdd

Merge branch 'master' of https://github.com/pingcap/tidb into impl_se…

4c0f77f

…lection

minor changes

2d761d1

eurekaka reviewed Oct 17, 2019

View reviewed changes

francis0407 added 2 commits October 18, 2019 14:14

address comment

1469db5

fix tests

0125747

francis0407 requested a review from eurekaka October 18, 2019 08:24

eurekaka approved these changes Oct 21, 2019

View reviewed changes

eurekaka added the status/can-merge Indicates a PR has been approved by a committer. label Oct 21, 2019

Merge branch 'master' into impl_selection

db20b66

Merge branch 'master' into impl_selection

84f0b80

sre-bot merged commit d0113f4 into pingcap:master Oct 21, 2019

francis0407 deleted the impl_selection branch October 29, 2019 02:43

XiaTianliang pushed a commit to XiaTianliang/tidb that referenced this pull request Dec 21, 2019

planner/cascades: implement ImplementationRule for Selection (pingcap…

ddc9a4e

…#12257)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

planner/cascades: implement ImplementationRule for Selection #12257

planner/cascades: implement ImplementationRule for Selection #12257

francis0407 commented Sep 18, 2019 •

edited

Loading

codecov bot commented Sep 18, 2019 •

edited

Loading

francis0407 commented Sep 18, 2019

francis0407 commented Sep 23, 2019

sre-bot commented Sep 25, 2019

sre-bot commented Sep 27, 2019

sre-bot commented Oct 7, 2019

winoros left a comment

sre-bot commented Oct 17, 2019

francis0407 commented Oct 17, 2019

francis0407 commented Oct 17, 2019

eurekaka Oct 17, 2019

francis0407 Oct 18, 2019

francis0407 Oct 18, 2019

eurekaka Oct 17, 2019

francis0407 Oct 18, 2019

eurekaka Oct 17, 2019

sre-bot commented Oct 20, 2019

eurekaka left a comment

sre-bot commented Oct 21, 2019

sre-bot commented Oct 21, 2019

planner/cascades: implement ImplementationRule for Selection #12257

planner/cascades: implement ImplementationRule for Selection #12257

Conversation

francis0407 commented Sep 18, 2019 • edited Loading

What problem does this PR solve?

Check List

codecov bot commented Sep 18, 2019 • edited Loading

Codecov Report

francis0407 commented Sep 18, 2019

francis0407 commented Sep 23, 2019

sre-bot commented Sep 25, 2019

sre-bot commented Sep 27, 2019

sre-bot commented Oct 7, 2019

winoros left a comment

Choose a reason for hiding this comment

sre-bot commented Oct 17, 2019

francis0407 commented Oct 17, 2019

francis0407 commented Oct 17, 2019

eurekaka Oct 17, 2019

Choose a reason for hiding this comment

francis0407 Oct 18, 2019

Choose a reason for hiding this comment

francis0407 Oct 18, 2019

Choose a reason for hiding this comment

eurekaka Oct 17, 2019

Choose a reason for hiding this comment

francis0407 Oct 18, 2019

Choose a reason for hiding this comment

eurekaka Oct 17, 2019

Choose a reason for hiding this comment

sre-bot commented Oct 20, 2019

eurekaka left a comment

Choose a reason for hiding this comment

sre-bot commented Oct 21, 2019

sre-bot commented Oct 21, 2019

francis0407 commented Sep 18, 2019 •

edited

Loading

codecov bot commented Sep 18, 2019 •

edited

Loading