Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plan: get correct columns #5818

Merged
merged 5 commits into from
Feb 11, 2018
Merged

plan: get correct columns #5818

merged 5 commits into from
Feb 11, 2018

Conversation

zimulala
Copy link
Contributor

@zimulala zimulala commented Feb 7, 2018

Use the correct columns in PhysicalIndexScan's ToPB function.
I need use the same columns in function buildDataSource and ToPB.

@zimulala zimulala added the WIP label Feb 7, 2018
@zimulala zimulala removed the WIP label Feb 8, 2018
@zimulala
Copy link
Contributor Author

zimulala commented Feb 8, 2018

PTAL @shenli @winkyao @winoros @coocood

@zimulala zimulala mentioned this pull request Feb 8, 2018
@zimulala zimulala added the type/bugfix This PR fixes a bug. label Feb 8, 2018
@shenli shenli added the priority/P1 The issue has P1 priority. label Feb 8, 2018
model/model.go Outdated
@@ -114,6 +114,9 @@ type TableInfo struct {
// Now it only uses for compatibility with the old version that already uses this field.
OldSchemaID int64 `json:"old_schema_id,omitempty"`

publicColumns []*ColumnInfo
writableColumns []*ColumnInfo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two fields are not needed.

model/model.go Outdated
@@ -172,6 +175,45 @@ func (t *TableInfo) GetPkColInfo() *ColumnInfo {
return nil
}

// Cols returns the columns of the table in public state.
func (t *TableInfo) Cols() []*ColumnInfo {
if len(t.publicColumns) > 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this field be modified by multiple goroutines?

func (s *testStateChangeSuite) TestUpdateOrDelete(c *C) {
sqls := make([]string, 2)
sqls[0] = "delete from t where c2 = 'a'"
sqls[1] = "update t set c2 = 'c2_update' where c2 = 'a'"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this update use index scan?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.
The delete SQL can't use index in a statement directly, so these two statements use drop state table to have the index scan.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add index hint to update statement.

"update t use index(c2) set c2 = 'c2_update' where c2 = 'a'"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So how to deal with delete statement?

@zimulala
Copy link
Contributor Author

zimulala commented Feb 8, 2018

/run-all-tests tidb-test=release-1.0 tidb-private-test=release-1.0 tikv=release-1.0

@zimulala
Copy link
Contributor Author

zimulala commented Feb 8, 2018

PTAL @shenli @coocood

_, err = se.Execute("use test_db_state")
c.Assert(err, IsNil)
callback.OnJobUpdatedExported = func(job *model.Job) {
if job.SchemaState == prevState || checkErr != nil || times >= 3 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why limit the times?

Copy link
Contributor Author

@zimulala zimulala Feb 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used for excluding the effects of other DDL statements.
Make the testing stable.

@coocood
Copy link
Member

coocood commented Feb 8, 2018

The original problem is the mismatch of the column position in slice and the column offset.

A cleaner way to fix this issue is to change createColumnInfo in ddl/column.go, we can append the non-public column to the end first, and when the column is public, change the position and offset of the newly added column.

@coocood
Copy link
Member

coocood commented Feb 8, 2018

If we append the new column to the end in non-public state, we don't need this PR #3754

@@ -89,14 +89,18 @@ func (p *PhysicalTableScan) ToPB(ctx context.Context) (*tipb.Executor, error) {
// ToPB implements PhysicalPlan ToPB interface.
func (p *PhysicalIndexScan) ToPB(ctx context.Context) (*tipb.Executor, error) {
columns := make([]*model.ColumnInfo, 0, p.schema.Len())
tableColumns := p.Table.Cols()
if ctx.GetSessionVars().StmtCtx.InUpdateStmt {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to check it and get writable columns because all the columns in an index must be public.

Copy link
Contributor Author

@zimulala zimulala Feb 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, now the column in an index must be public.

@@ -48,6 +48,7 @@ func (s *testStateChangeSuite) SetUpSuite(c *C) {
s.store, err = tikv.NewMockTikvStore()
c.Assert(err, IsNil)
tidb.SetSchemaLease(s.lease)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove line 50?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

never mind, I read it wrong.

@zimulala
Copy link
Contributor Author

zimulala commented Feb 9, 2018

@coocood
I see. But the PR change a little. If update createColumnInfo, I'm afraid the impact of larger. I hope to solve this bug first, then change createColumnInfo.

@coocood
Copy link
Member

coocood commented Feb 9, 2018

LGTM

Copy link
Contributor

@winkyao winkyao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@winkyao
Copy link
Contributor

winkyao commented Feb 9, 2018

@shenli PTAL

@zimulala zimulala added the status/LGT2 Indicates that a PR has LGTM 2. label Feb 9, 2018
@@ -172,6 +172,22 @@ func (t *TableInfo) GetPkColInfo() *ColumnInfo {
return nil
}

// Cols returns the columns of the table in public state.
func (t *TableInfo) Cols() []*ColumnInfo {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation is not efficient. Could you do some pre-calculation?

@shenli
Copy link
Member

shenli commented Feb 11, 2018

LGTM
We need to optimize this latter.

@shenli shenli merged commit fafc39d into pingcap:release-1.0 Feb 11, 2018
@zimulala zimulala deleted the fix-columns branch August 4, 2022 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority/P1 The issue has P1 priority. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants