Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner/core: get back range columns pruning after last refactor #15169

Merged
merged 3 commits into from
Mar 18, 2020

Conversation

tiancaiamao
Copy link
Contributor

@tiancaiamao tiancaiamao commented Mar 6, 2020

What problem does this PR solve?

create table t (a varchar(32)) partition by range columns (a) (
    partition p0 values less than ('c'),
    partition p1 values less than ('f'),
    partition p2 values less than ('g'),
    partition p3 values less than ('n'),
    partition p4 values less than ('z')
);
select * from t;
Error Code: 1105. strconv.ParseInt: parsing "\"c\"": invalid syntax

What is changed and how it works?

Fix regression after #14834

Partition pruning for 'partition by range columns' table is broken after that refactor, as we only handle integer in the new partition pruning algorithm, 'by range columns' may contain non-integer data.

Now I've added expression comparing in addition to integer in this commit.

Check List

Tests

  • Unit test

@tiancaiamao tiancaiamao added the sig/planner SIG: Planner label Mar 6, 2020
@tiancaiamao tiancaiamao requested a review from a team as a code owner March 6, 2020 06:02
@ghost ghost requested review from eurekaka and francis0407 and removed request for a team March 6, 2020 06:03
@tiancaiamao
Copy link
Contributor Author

PTAL @imtbkcat @lysu

@codecov
Copy link

codecov bot commented Mar 6, 2020

Codecov Report

Merging #15169 into master will decrease coverage by 0.0902%.
The diff coverage is 85.1239%.

@@               Coverage Diff                @@
##             master     #15169        +/-   ##
================================================
- Coverage   80.3669%   80.2766%   -0.0903%     
================================================
  Files           502        503         +1     
  Lines        133805     132280      -1525     
================================================
- Hits         107535     106190      -1345     
+ Misses        17823      17700       -123     
+ Partials       8447       8390        -57

@imtbkcat imtbkcat self-requested a review March 9, 2020 04:08
@lysu lysu self-requested a review March 9, 2020 08:46
planner/core/rule_partition_processor.go Outdated Show resolved Hide resolved
planner/core/rule_partition_processor.go Outdated Show resolved Hide resolved
col, fn, err = makePartitionByFnCol(ds.ctx, ds.TblCols, ds.names, pi.Columns[0].L)
// Partition by range columns.
if len(pi.Columns) > 0 {
return s.pruneRangeColumnsPartition(ds, pi)
Copy link
Contributor

@lysu lysu Mar 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we let "range column by int" like this can use makeLessThanData?

create table t (a int) partition by range columns (a) (
    partition p0 values less than (0),
    partition p1 values less than (500),
    partition p2 values less than (1000),
    partition p3 values less than (1500),
    partition p4 values less than (2000)
);
select * from t;

(that way should be faster?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's faster, but the case is not general, we have to write more code for this optimizing

}
}
var expr expression.Expression
expr, err = expression.NewFunction(sctx, op, types.NewFieldType(mysql.TypeLonglong), p.data[ith], v)
Copy link
Contributor

@lysu lysu Mar 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both RANGE COLUMNS partitioning and LIST COLUMNS partitioning support the use of non-integer columns for defining value ranges or list members. The permitted data types are shown in the following list:

  • All integer types: TINYINT, SMALLINT, MEDIUMINT, INT (INTEGER), and BIGINT. (This is the same as with partitioning by RANGE and LIST.) Other numeric data types (such as DECIMAL or FLOAT) are not supported as partitioning columns.

  • DATE and DATETIME. Columns using other data types relating to dates or times are not supported as partitioning columns.

  • The following string types: CHAR, VARCHAR, BINARY, and VARBINARY. TEXT and BLOB columns are not supported as partitioning columns.

it seems partition by range columns only support three type int/date/string, so ideal way is impl two addition lessThanData type ---------- lessThanDataInt, lessThanDataTime and lessThanDataString to void new & eval expression for each compare?

but LGTM to use current impl to quick fix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If some type is not supported, we should prevent the case in DDL check, rather than in partition pruning.
lessThanDataTime and lessThanDataString seems good, we can optimize it later.

@lysu
Copy link
Contributor

lysu commented Mar 10, 2020

@winoros please help to take a look if free

@tiancaiamao
Copy link
Contributor Author

/rebuild

@lysu
Copy link
Contributor

lysu commented Mar 12, 2020

LGTM, @winoros @eurekaka PTAL~

@lysu lysu added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 12, 2020
@tiancaiamao
Copy link
Contributor Author

@imtbkcat PTAL

Copy link

@imtbkcat imtbkcat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tiancaiamao
Copy link
Contributor Author

/merge

@sre-bot sre-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 18, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Mar 18, 2020

/run-all-tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants