Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner/cascades: implement ImplementationRule for Selection #12257

Merged
merged 28 commits into from
Oct 21, 2019
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
00b8a33
implement ImplementationRule for Selection
francis0407 Sep 18, 2019
8d261fb
address comment
francis0407 Sep 18, 2019
b32365d
revert change
francis0407 Sep 18, 2019
4f61d6c
Split SelectionImpl into RootSelectionImpl and TiKVCopSelectionImpl.
francis0407 Sep 23, 2019
151182a
add comments for EngineTypes
francis0407 Sep 23, 2019
623a8bb
add test case for root selection
francis0407 Sep 24, 2019
90c3f43
Merge branch 'master' of https://github.com/pingcap/tidb into impl_se…
francis0407 Sep 26, 2019
9d1cf40
Merge branch 'master' of https://github.com/pingcap/tidb into impl_se…
francis0407 Sep 26, 2019
df118b1
add engine type in pattern
francis0407 Sep 26, 2019
5b2f2d4
Merge remote-tracking branch 'origin/impl_selection' into impl_selection
francis0407 Sep 26, 2019
95374f2
Merge branch 'impl_selection' of https://github.com/francis0407/tidb …
francis0407 Sep 26, 2019
0e0ffda
fix comment
francis0407 Sep 26, 2019
768dd21
fix implementation name
francis0407 Sep 29, 2019
2af3065
simplify engine type check in exprIter
francis0407 Sep 29, 2019
350887b
Merge branch 'impl_selection' of https://github.com/francis0407/tidb …
francis0407 Sep 29, 2019
bf062f7
CRLF->LF?
francis0407 Sep 29, 2019
734764f
replace (Operand, EngineTypeSet) as *Pattern in ExprIter
francis0407 Sep 30, 2019
5f19760
fix the comment
francis0407 Sep 30, 2019
9c5bf85
Merge branch 'master' of https://github.com/pingcap/tidb into impl_se…
francis0407 Oct 12, 2019
f3d2def
resolve conflicts
francis0407 Oct 12, 2019
78aca89
Merge remote-tracking branch 'origin/impl_selection' into impl_selection
francis0407 Oct 12, 2019
4e52cdd
Add a test case for the pattern without a leaf AnyOperand
francis0407 Oct 15, 2019
4c0f77f
Merge branch 'master' of https://github.com/pingcap/tidb into impl_se…
francis0407 Oct 17, 2019
2d761d1
minor changes
francis0407 Oct 17, 2019
1469db5
address comment
francis0407 Oct 18, 2019
0125747
fix tests
francis0407 Oct 18, 2019
db20b66
Merge branch 'master' into impl_selection
sre-bot Oct 21, 2019
84f0b80
Merge branch 'master' into impl_selection
sre-bot Oct 21, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions planner/cascades/implementation_rules.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@ var implementationMap = map[memo.Operand][]ImplementationRule{
memo.OperandShow: {
&ImplShow{},
},
memo.OperandSelection: {
&ImplSelection{},
},
}

// ImplTableDual implements LogicalTableDual as PhysicalTableDual.
Expand Down Expand Up @@ -164,3 +167,21 @@ func (r *ImplShow) OnImplement(expr *memo.GroupExpr, reqProp *property.PhysicalP
showPhys.SetSchema(logicProp.Schema)
return impl.NewShowImpl(showPhys), nil
}

// ImplSelection is the implementation rule which implements LogicalSelection to PhysicalSelection.
type ImplSelection struct {
}

// Match implements ImplementationRule Match interface.
func (r *ImplSelection) Match(expr *memo.GroupExpr, prop *property.PhysicalProperty) (matched bool) {
return true
francis0407 marked this conversation as resolved.
Show resolved Hide resolved
}

// OnImplement implements ImplementationRule OnImplement interface.
func (r *ImplSelection) OnImplement(expr *memo.GroupExpr, reqProp *property.PhysicalProperty) (memo.Implementation, error) {
logicalSel := expr.ExprNode.(*plannercore.LogicalSelection)
physicalSel := plannercore.PhysicalSelection{
Conditions: logicalSel.Conditions,
}.Init(logicalSel.SCtx(), expr.Group.Prop.Stats.ScaleByExpectCnt(reqProp.ExpectedCnt), logicalSel.SelectBlockOffset(), reqProp)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we make a copy of the reqProp to pass it as children properties? see #8635 for explanations.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I just find another problem.
The Stats for init should be expr.Group.Prop.Stats instead of expr.Group.Prop.Stats.ScaleByExpectCnt(reqProp.ExpectedCnt)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And reqProp.ExpectedCnt also should be scaled by the selectivity?

return impl.NewSelectionImpl(physicalSel), nil
}
37 changes: 21 additions & 16 deletions planner/cascades/integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,14 @@ import (
"github.com/pingcap/tidb/session"
"github.com/pingcap/tidb/store/mockstore"
"github.com/pingcap/tidb/util/testkit"
"github.com/pingcap/tidb/util/testutil"
)

var _ = Suite(&testIntegrationSuite{})

type testIntegrationSuite struct {
store kv.Storage
store kv.Storage
testData testutil.TestData
}

func newStoreWithBootstrap() (kv.Storage, error) {
Expand All @@ -40,9 +42,12 @@ func (s *testIntegrationSuite) SetUpSuite(c *C) {
var err error
s.store, err = newStoreWithBootstrap()
c.Assert(err, IsNil)
s.testData, err = testutil.LoadTestSuiteData("testdata", "integration_suite")
c.Assert(err, IsNil)
}

func (s *testIntegrationSuite) TearDownSuite(c *C) {
c.Assert(s.testData.GenerateOutputIfNeeded(), IsNil)
s.store.Close()
}

Expand All @@ -62,22 +67,22 @@ func (s *testIntegrationSuite) TestPKIsHandleRangeScan(c *C) {
tk := testkit.NewTestKitWithInit(c, s.store)
tk.MustExec("drop table if exists t")
tk.MustExec("create table t(a int primary key, b int)")
tk.MustExec("insert into t values(1,2),(3,4)")
tk.MustExec("insert into t values(1,2),(3,4),(5,6)")
tk.MustExec("set session tidb_enable_cascades_planner = 1")
tk.MustQuery("explain select b from t where a > 1").Check(testkit.Rows(
"Projection_8 3333.33 root Column#2",
"└─TableReader_9 3333.33 root data:TableScan_10",
" └─TableScan_10 3333.33 cop table:t, range:(1,+inf], keep order:false, stats:pseudo",
))
tk.MustQuery("select b from t where a > 1").Check(testkit.Rows(
"4",
))
tk.MustQuery("explain select b from t where a > 1 and a < 3").Check(testkit.Rows(
"Projection_8 2.00 root Column#2",
"└─TableReader_9 2.00 root data:TableScan_10",
" └─TableScan_10 2.00 cop table:t, range:(1,3), keep order:false, stats:pseudo",
))
tk.MustQuery("select b from t where a > 1 and a < 3").Check(testkit.Rows())

var input []string
var output []struct {
SQL string
Result []string
}
s.testData.GetTestCases(c, &input, &output)
for i, sql := range input {
s.testData.OnRecord(func() {
output[i].SQL = sql
output[i].Result = s.testData.ConvertRowsToStrings(tk.MustQuery(sql).Rows())
})
tk.MustQuery(sql).Check(testkit.Rows(output[i].Result...))
}
}

func (s *testIntegrationSuite) TestBasicShow(c *C) {
Expand Down
15 changes: 15 additions & 0 deletions planner/cascades/testdata/integration_suite_in.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[
{
"name": "TestPKIsHandleRangeScan",
"cases": [
"explain select b from t where a > 1",
"select b from t where a > 1",
"explain select b from t where a > 1 and a < 3",
"select b from t where a > 1 and a < 3",
"explain select b from t where a > 1 and b < 6",
"select b from t where a > 1 and b < 6",
"explain select a from t where a * 3 + 1 > 9 and a < 5",
"select a from t where a * 3 + 1 > 9 and a < 5"
]
}
]
64 changes: 64 additions & 0 deletions planner/cascades/testdata/integration_suite_out.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
[
{
"Name": "TestPKIsHandleRangeScan",
"Cases": [
{
"SQL": "explain select b from t where a > 1",
"Result": [
"Projection_8 3333.33 root Column#2",
"└─TableReader_9 3333.33 root data:TableScan_10",
" └─TableScan_10 3333.33 cop table:t, range:(1,+inf], keep order:false, stats:pseudo"
]
},
{
"SQL": "select b from t where a > 1",
"Result": [
"4",
"6"
]
},
{
"SQL": "explain select b from t where a > 1 and a < 3",
"Result": [
"Projection_8 2.00 root Column#2",
"└─TableReader_9 2.00 root data:TableScan_10",
" └─TableScan_10 2.00 cop table:t, range:(1,3), keep order:false, stats:pseudo"
]
},
{
"SQL": "select b from t where a > 1 and a < 3",
"Result": null
},
{
"SQL": "explain select b from t where a > 1 and b < 6",
"Result": [
"Projection_9 2666.67 root Column#2",
"└─TableReader_10 2666.67 root data:Selection_11",
" └─Selection_11 2666.67 cop lt(Column#2, 6)",
" └─TableScan_12 3333.33 cop table:t, range:(1,+inf], keep order:false, stats:pseudo"
]
},
{
"SQL": "select b from t where a > 1 and b < 6",
"Result": [
"4"
]
},
{
"SQL": "explain select a from t where a * 3 + 1 > 9 and a < 5",
"Result": [
"Projection_9 4.00 root Column#1",
"└─TableReader_10 4.00 root data:Selection_11",
" └─Selection_11 4.00 cop gt(plus(mul(Column#1, 3), 1), 9)",
" └─TableScan_12 5.00 cop table:t, range:[-inf,5), keep order:false, stats:pseudo"
]
},
{
"SQL": "select a from t where a * 3 + 1 > 9 and a < 5",
"Result": [
"3"
]
}
]
}
]
20 changes: 19 additions & 1 deletion planner/implementation/simple_plans.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@ package implementation

import (
plannercore "github.com/pingcap/tidb/planner/core"
"github.com/pingcap/tidb/planner/memo"
)

// ProjectionImpl implementation of PhysicalProjection.
// ProjectionImpl is the implementation of PhysicalProjection.
type ProjectionImpl struct {
baseImpl
}
Expand All @@ -36,3 +37,20 @@ type ShowImpl struct {
func NewShowImpl(show *plannercore.PhysicalShow) *ShowImpl {
return &ShowImpl{baseImpl: baseImpl{plan: show}}
}

// SelectionImpl is the implementation of PhysicalSelection.
type SelectionImpl struct {
baseImpl
}

// CalcCost implements Implementation CalcCost interface.
func (sel *SelectionImpl) CalcCost(outCount float64, childCosts []float64, children ...*memo.Group) float64 {
// TODO: When CPUFactor and CopCPUFactor can be set by users, we need to distinguish them.
zz-jason marked this conversation as resolved.
Show resolved Hide resolved
sel.cost = outCount*plannercore.CPUFactor + childCosts[0]
return sel.cost
}

// NewSelectionImpl creates a new SelectionImpl.
func NewSelectionImpl(sel *plannercore.PhysicalSelection) *SelectionImpl {
return &SelectionImpl{baseImpl{plan: sel}}
}