Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

infoschema,planner: implement the CLUSTER_LOG memory table predicate push down #14018

Merged
merged 18 commits into from
Dec 16, 2019
Merged

infoschema,planner: implement the CLUSTER_LOG memory table predicate push down #14018

merged 18 commits into from
Dec 16, 2019

Conversation

lonng
Copy link
Contributor

@lonng lonng commented Dec 11, 2019

Signed-off-by: Lonng heng@lonng.org

What problem does this PR solve?

This PR implement the predicate push-down part of the CLUSTER_LOG memory table.
It's a subtask of issue #13567

What is changed and how it works?

  1. Add the CLUSTER_LOG memory table.
  2. Implement the ClusterLogTableExtractor

NOTE: The CLUSTER_LOG memory table implementation will be split into two PRs to make it reviewable. The retriever will be implemented in the next PR.

Check List

Tests

  • Unit test

Release note

  • No need.

Signed-off-by: Lonng <heng@lonng.org>
@lonng lonng requested a review from a team as a code owner December 11, 2019 03:31
@lonng lonng added the type/enhancement The issue or PR belongs to an enhancement. label Dec 11, 2019
@ghost ghost requested review from alivxxx and winoros and removed request for a team December 11, 2019 03:31
@codecov
Copy link

codecov bot commented Dec 11, 2019

Codecov Report

Merging #14018 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #14018   +/-   ##
===========================================
  Coverage   80.2791%   80.2791%           
===========================================
  Files           483        483           
  Lines        122353     122353           
===========================================
  Hits          98224      98224           
  Misses        16366      16366           
  Partials       7763       7763

Signed-off-by: Lonng <heng@lonng.org>
infoschema/tables.go Show resolved Hide resolved
planner/core/memtable_predicate_extractor.go Outdated Show resolved Hide resolved

// extracts the time range column, e.g:
// SELECT * FROM t WHERE time='2019-10-10 10:10:10'
// SELECT * FROM t WHERE time>'2019-10-10 10:10:10' AND time<'2019-10-11 10:10:10'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you don't deal with (time > ... and time < ...) or (time > ... and time < ...)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, to simplify the implementation of the diagnostics service search log interface, it can just accept only one range of time (pick a group of the log files in the time range). So the filtering will be executed in TiDB via Selection executor.

Copy link
Contributor Author

@lonng lonng Dec 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, we use the cluster level log search function to locate the problem usually only one range of time. So I think it's enough to cover this scenario.

Copy link
Contributor

@crazycs520 crazycs520 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

Signed-off-by: Lonng <heng@lonng.org>
Signed-off-by: Lonng <heng@lonng.org>
Copy link
Contributor

@crazycs520 crazycs520 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@djshow832
Copy link
Contributor

Please modify the PR title: infoschema, planner: xxx
Rest LGTM

@lonng lonng changed the title implement the CLUSTER_LOG memory table predicate push down planner: implement the CLUSTER_LOG memory table predicate push down Dec 16, 2019
@lonng lonng changed the title planner: implement the CLUSTER_LOG memory table predicate push down infoschema,planner: implement the CLUSTER_LOG memory table predicate push down Dec 16, 2019
@lonng
Copy link
Contributor Author

lonng commented Dec 16, 2019

@djshow832 Done

Copy link
Member

@winoros winoros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@crazycs520 crazycs520 added the status/can-merge Indicates a PR has been approved by a committer. label Dec 16, 2019
@sre-bot
Copy link
Contributor

sre-bot commented Dec 16, 2019

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Dec 16, 2019

@lonng merge failed.

@lonng
Copy link
Contributor Author

lonng commented Dec 16, 2019

/run-unit-test

@lonng lonng merged commit 7262383 into pingcap:master Dec 16, 2019
@lonng lonng deleted the cluster-log-push-down branch December 16, 2019 13:53
@lonng lonng added sig/planner SIG: Planner status/LGT2 Indicates that a PR has LGTM 2. labels Dec 17, 2019
XiaTianliang pushed a commit to XiaTianliang/tidb that referenced this pull request Dec 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants