Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: implement the INSPECTION_SCHEMA to provide snapshot of inspection tables #14147

Merged
merged 19 commits into from
Dec 24, 2019
Merged

*: implement the INSPECTION_SCHEMA to provide snapshot of inspection tables #14147

merged 19 commits into from
Dec 24, 2019

Conversation

lonng
Copy link
Contributor

@lonng lonng commented Dec 19, 2019

Signed-off-by: Lonng heng@lonng.org

What problem does this PR solve?

This PR introduces a new virtual system schema inspection_schema, which is used to provide a consistent view of information_schema tables, the related table will have the same table name within information_schema. The data will be obtained lazily from information_schema and cache in SessionVars, and the cached data will be cleared at InspectionExec closing.

This PR is part of #13567

The reason why we should cache the data because some data of cluster-level memory tables will be retrieved many times in different inspection rules, and the cost of retrieving some data is expensive. We use the TableSnapshot to cache those data.

What is changed and how it works?

  1. Add a new struct TableSnapshot, which represents a data snapshot of the table contained in inspection_schema.
  2. Add a new system schema inspection_schema.
  3. Obtain data from information_schema lazily and cache them.

Check List

Tests

  • Unit test

Release note

  • Introduces a new virtual system schema, which represents some inspection data snapshots and valid in the lifetime of InspectionExec executor lives.

…tables

Signed-off-by: Lonng <heng@lonng.org>
Signed-off-by: Lonng <heng@lonng.org>
Signed-off-by: Lonng <heng@lonng.org>
Signed-off-by: Lonng <heng@lonng.org>
@zz-jason
Copy link
Member

@lonng please add some proper labels.

util/misc.go Outdated Show resolved Hide resolved
infoschema/inspection_schema_test.go Show resolved Hide resolved
Signed-off-by: Lonng <heng@lonng.org>
Signed-off-by: Lonng <heng@lonng.org>
@lonng
Copy link
Contributor Author

lonng commented Dec 20, 2019

/run-all-tests tidb-test=pr/970

infoschema/inspection_schema.go Outdated Show resolved Hide resolved
infoschema/inspection_schema.go Outdated Show resolved Hide resolved
infoschema/inspection_schema.go Outdated Show resolved Hide resolved
infoschema/inspection_schema.go Show resolved Hide resolved
infoschema/inspection_schema.go Show resolved Hide resolved
@@ -42,6 +42,8 @@ const (
PerformanceSchemaDBID int64 = SystemSchemaIDFlag | 10000
// MetricSchemaDBID is the metric_schema schema id, it's exported for test.
MetricSchemaDBID int64 = SystemSchemaIDFlag | 20000
// InspectionSchemaDBID is the inspection_schema id, it's exports for test.
InspectionSchemaDBID int64 = SystemSchemaIDFlag | 30000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard allocation code here should rebase autoID? if not, what if successive user creates schema/table with the same autoID.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All id which allocated in SystemSchemaIDFlag namespace will not conflict with tables created by the user.

Copy link
Contributor

@AilinKid AilinKid Dec 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even if the same ID is used both in SystemSchemaIDFlag and common table?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the common table ID starts from 0 and will never conflict with the system table.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See.

Copy link
Contributor

@djshow832 djshow832 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lonng
Copy link
Contributor Author

lonng commented Dec 24, 2019

/run-all-tests tidb-test=pr/970

Copy link
Contributor

@AilinKid AilinKid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@AilinKid AilinKid added the status/LGT2 Indicates that a PR has LGTM 2. label Dec 24, 2019
@lonng
Copy link
Contributor Author

lonng commented Dec 24, 2019

/run-all-tests tidb-test=pr/970

AilinKid
AilinKid previously approved these changes Dec 24, 2019
@AilinKid
Copy link
Contributor

AilinKid commented Dec 24, 2019

need fix the check dev and test first.

Signed-off-by: Lonng <heng@lonng.org>
@lonng
Copy link
Contributor Author

lonng commented Dec 24, 2019

/run-all-tests tidb-test=pr/970

@lonng
Copy link
Contributor Author

lonng commented Dec 24, 2019

/run-unit-test

Copy link
Contributor

@djshow832 djshow832 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve

@lonng lonng merged commit e8c198d into pingcap:master Dec 24, 2019
@lonng lonng deleted the inspection-schema branch December 24, 2019 03:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants