Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ddl: fix corrupted default value for bit type column (#18036) #20340

Merged
merged 6 commits into from
Oct 22, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #18036 to release-4.0


What problem does this PR solve?

Issue Number: close #17641 close #17642

Problem Summary:

The ColumnInfo.OriginDefaultValue is corrupted when the column type is BIT. BIT type column values are stored as strings by JSON marshaling and read it by unmarshaling later. However, the data in a string can be changed in this case:

originBytes := []byte{25, 185}
originStr := string(originBytes)

mr, _ := json.Marshal(originStr)
var jsoned string
_ = json.Unmarshal(mr, &jsoned)

fmt.Printf("%v\n", []byte(originStr))
fmt.Printf("%v\n", []byte(jsoned))
[25 185]
[25 239 191 189]

If we store it as []byte, the problem disappeared. This is how DefaultValueBit works.

type ColumnInfo struct {
	// ...
	OriginDefaultValue    interface{} `json:"origin_default"`
	DefaultValue          interface{} `json:"default"`
	DefaultValueBit       []byte      `json:"default_bit"`
	// ...
}

The same problem exists in OriginDefaultValue. PR #12168 tried to restore the OriginDefaultValue from DefaultValueBit in the getter, but OriginDefaultValue is different from DefaultValue: OriginDefaultValue is used to skip the backfilling phase of AddColumn(). If a column's default value is modified by ALTER TABLE MODIFY COLUMN DEFAULT, they are not equal anymore(see the example in #17641).

What is changed and how it works?

This PR adds a field OriginDefaultValueBit to ColumnInfo, which borrows from DefaultValueBit to solve the problem completely.

Proposal: xxx

What's Changed:

How it Works:
If the column type is BIT, set/get the content with OriginDefaultValueBit.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test

Side effects

Release note

  • fix corrupted default value for bit type column.

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot added sig/sql-infra SIG: SQL Infra type/4.0-cherry-pick type/bugfix This PR fixes a bug. labels Oct 4, 2020
@ti-srebot ti-srebot added this to the v4.0.8 milestone Oct 4, 2020
@djshow832 djshow832 removed their request for review October 13, 2020 08:22
@tangenta tangenta added the priority/release-blocker This issue blocks a release. Please solve it ASAP. label Oct 14, 2020
@tangenta
Copy link
Contributor

/run-all-tests

@tangenta
Copy link
Contributor

/run-all-tests

@tangenta
Copy link
Contributor

Test Failed Log:

[2020-10-22T09:07:01.872Z] java.net.SocketTimeoutException: sent ping but didn't receive pong within 30000ms (after 4 successful ping/pongs)
[2020-10-22T09:07:01.872Z] 	at okhttp3.internal.ws.RealWebSocket.writePingFrame(RealWebSocket.java:546)
[2020-10-22T09:07:01.872Z] 	at okhttp3.internal.ws.RealWebSocket$PingRunnable.run(RealWebSocket.java:530)
[2020-10-22T09:07:01.872Z] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2020-10-22T09:07:01.872Z] 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[2020-10-22T09:07:01.873Z] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[2020-10-22T09:07:01.873Z] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[2020-10-22T09:07:01.873Z] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2020-10-22T09:07:01.873Z] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2020-10-22T09:07:01.873Z] 	at java.lang.Thread.run(Thread.java:748)
[2020-10-22T09:07:08.759Z] Cannot contact test-go1130-memvolume-m5m30: hudson.remoting.RequestAbortedException: java.nio.channels.ClosedChannelException
[2020-10-22T09:14:01.683Z] Could not connect to test-go1130-memvolume-m5m30 to send interrupt signal to process

@tangenta
Copy link
Contributor

/run-all-tests

Copy link
Contributor

@AilinKid AilinKid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 22, 2020
Copy link
Member

@wjhuang2016 wjhuang2016 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 22, 2020
@bb7133
Copy link
Member

bb7133 commented Oct 22, 2020

/run-sqllogic-test
/run-unit-test

@ti-srebot ti-srebot added status/LGT3 The PR has already had 3 LGTM. and removed status/LGT2 Indicates that a PR has LGTM 2. labels Oct 22, 2020
@bb7133 bb7133 merged commit a132f2b into pingcap:release-4.0 Oct 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority/release-blocker This issue blocks a release. Please solve it ASAP. sig/sql-infra SIG: SQL Infra status/LGT3 The PR has already had 3 LGTM. type/bugfix This PR fixes a bug. type/4.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants