Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

*: add executor runtime info for explain for connection statement (#19183) #20384

Merged
merged 7 commits into from
Oct 22, 2020
Merged

*: add executor runtime info for explain for connection statement (#19183) #20384

merged 7 commits into from
Oct 22, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #19183 to release-4.0

What problem does this PR solve?

Issue Number: close #18233

Problem Summary:
add this runtime information in explain for connection statement.

Example:

When set @@tidb_enable_collect_execution_info=1;

test> show processlist;
+-----+------+-----------+------+---------+------+------------+-----------------------------------------------------------+
| Id  | User | Host      | db   | Command | Time | State      | Info                                                      |
+-----+------+-----------+------+---------+------+------------+-----------------------------------------------------------+
| 775 | root | 127.0.0.1 | test | Query   | 65   | autocommit | select count(*) from t1 tt1 join t1 tt2 where tt1.a>tt2.a |
| 1   | root | 127.0.0.1 | test | Query   | 0    | autocommit | show processlist                                          |
+-----+------+-----------+------+---------+------+------------+-----------------------------------------------------------+
test> explain for connection 775;
+---------------------------+-------------+---------+-----------+-----------------------+----------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| id                        | estRows     | actRows | task      | access object         | execution info                                                                                                       | operator info                                             | memory | disk |
+---------------------------+-------------+---------+-----------+-----------------------+----------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| HashAgg_11                | 1.00        | 0       | root      |                       | time:0s, loops:0                                                                                                     | funcs:count(1)->Column#7                                  | N/A    | N/A  |
| └─HashJoin_14             | 99800100.00 | 1083392 | root      |                       | time:11.67287184s, loops:1058                                                                                        | CARTESIAN inner join, other cond:gt(test.t1.a, test.t1.a) | N/A    | N/A  |
|   ├─IndexReader_19(Build) | 9990.00     | 482836  | root      |                       | time:138.962375ms, loops:473, cop_task: {num: 1, max:125.755703ms, proc_keys: 0, rpc_num: 1, rpc_time: 125.729723ms} | index:IndexFullScan_18                                    | N/A    | N/A  |
|   │ └─IndexFullScan_18    | 9990.00     | 0       | cop[tikv] | table:tt2, index:a(a) | time:0ns, loops:0                                                                                                    | keep order:false, stats:pseudo                            | N/A    | N/A  |
|   └─IndexReader_17(Probe) | 9990.00     | 5120    | root      |                       | time:126.260071ms, loops:5, cop_task: {num: 1, max:125.596953ms, proc_keys: 0, rpc_num: 1, rpc_time: 125.582473ms}   | index:IndexFullScan_16                                    | N/A    | N/A  |
|     └─IndexFullScan_16    | 9990.00     | 0       | cop[tikv] | table:tt1, index:a(a) | time:0ns, loops:0                                                                                                    | keep order:false, stats:pseudo                            | N/A    | N/A  |
+---------------------------+-------------+---------+-----------+-----------------------+----------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+

# after a while
test> explain for connection 775;
+---------------------------+-------------+-----------+-----------+-----------------------+----------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| id                        | estRows     | actRows   | task      | access object         | execution info                                                                   | operator info                                             | memory | disk |
+---------------------------+-------------+-----------+-----------+-----------------------+----------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| HashAgg_11                | 1.00        | 0         | root      |                       | time:0s, loops:0                                                                 | funcs:count(1)->Column#7                                  | N/A    | N/A  |
| └─HashJoin_14             | 99800100.00 | 145286144 | root      |                       | time:6m3.062011395s, loops:141881                                                | CARTESIAN inner join, other cond:gt(test.t1.a, test.t1.a) | N/A    | N/A  |
|   ├─IndexReader_19(Build) | 9990.00     | 482836    | root      |                       | time:138.962375ms, loops:473, cop_task: {num: 1, max:125.755703ms, proc_keys: 0} | index:IndexFullScan_18                                    | N/A    | N/A  |
|   │ └─IndexFullScan_18    | 9990.00     | 0         | cop[tikv] | table:tt2, index:a(a) | time:0ns, loops:0                                                                | keep order:false, stats:pseudo                            | N/A    | N/A  |
|   └─IndexReader_17(Probe) | 9990.00     | 19456     | root      |                       | time:127.068821ms, loops:19, cop_task: {num: 1, max:125.596953ms, proc_keys: 0}  | index:IndexFullScan_16                                    | N/A    | N/A  |
|     └─IndexFullScan_16    | 9990.00     | 0         | cop[tikv] | table:tt1, index:a(a) | time:0ns, loops:0                                                                | keep order:false, stats:pseudo                            | N/A    | N/A  |
+---------------------------+-------------+-----------+-----------+-----------------------+----------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+

When set @@tidb_enable_collect_execution_info=0;, Since TiDB doesn't collect runtime information, so explan for connection will also not display the executor info.

explain for connection 778;
+---------------------------+-----------------+-----------+-----------------------+-----------------------------------------------------------+
| id                        | estRows         | task      | access object         | operator info                                             |
+---------------------------+-----------------+-----------+-----------------------+-----------------------------------------------------------+
| HashAgg_11                | 1.00            | root      |                       | funcs:count(1)->Column#7                                  |
| └─HashJoin_14             | 233130602896.00 | root      |                       | CARTESIAN inner join, other cond:gt(test.t1.a, test.t1.a) |
|   ├─IndexReader_19(Build) | 482836.00       | root      |                       | index:IndexFullScan_18                                    |
|   │ └─IndexFullScan_18    | 482836.00       | cop[tikv] | table:tt2, index:a(a) | keep order:false                                          |
|   └─IndexReader_17(Probe) | 482836.00       | root      |                       | index:IndexFullScan_16                                    |
|     └─IndexFullScan_16    | 482836.00       | cop[tikv] | table:tt1, index:a(a) | keep order:false                                          |
+---------------------------+-----------------+-----------+-----------------------+-----------------------------------------------------------+

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM
  • Breaking backward compatibility

Release note

  • Add executor information in explain for connection result.

@hesper-guo @ti-srebot
cherry pick pingcap#19183 to release-4.0
c022ad3 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot mentioned this pull request Oct 10, 2020
Merged
@ti-srebot ti-srebot added this to the v4.0.8 milestone Oct 10, 2020
@ti-srebot ti-srebot self-assigned this Oct 10, 2020
@crazycs520 crazycs520 self-assigned this Oct 10, 2020
qw4990
qw4990 reviewed Oct 19, 2020
View reviewed changes
executor/prepared_test.go Outdated Show resolved Hide resolved
@crazycs520
Copy link
Contributor

/run-all-tests

@crazycs520
Copy link
Contributor

/run-all-tests

SunRunAway
SunRunAway approved these changes Oct 22, 2020
View reviewed changes
Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 22, 2020
@SunRunAway
Copy link
Contributor

@qw4990 PTAL

@SunRunAway
Copy link
Contributor

@crazycs520 documentation is needed.

@crazycs520
Copy link
Contributor

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Oct 22, 2020
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot merged commit fdd7a42 into pingcap:release-4.0 Oct 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qw4990 qw4990 qw4990 left review comments

@SunRunAway SunRunAway SunRunAway approved these changes

@crazycs520 crazycs520 Awaiting requested review from crazycs520

@lzmhhh123 lzmhhh123 Awaiting requested review from lzmhhh123

Assignees

@crazycs520 crazycs520

@ti-srebot ti-srebot

Labels
contribution This PR is from a community contributor. sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1. type/4.0-cherry-pick
Projects
None yet
Milestone
v4.0.8
Development

Successfully merging this pull request may close these issues.
5 participants
@ti-srebot @crazycs520 @SunRunAway @qw4990 @hesper-guo