Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support ALTER TABLE t REORGANIZE PARTITION #8114

Merged
merged 9 commits into from
Mar 16, 2023
Merged

Support ALTER TABLE t REORGANIZE PARTITION #8114

merged 9 commits into from
Mar 16, 2023

Conversation

mjonss
Copy link
Contributor

@mjonss mjonss commented Jan 25, 2023

What problem does this PR solve?

Issue Number: close #8113

What is changed and how it works?

Adding timodel.ActionReorganizePartition to the allowDDLList and include it for updatePartition in DoHandleDDL.

Check List

Tests

  • Unit test
  • Integration test

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note 

Adding support for ALTER TABLE ... REORGANIZE PARTITION

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jan 25, 2023 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • asddongmen
  • zhaoxinyu

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jan 25, 2023
@sre-bot
Copy link

sre-bot commented Jan 25, 2023 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@ti-chi-bot ti-chi-bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jan 25, 2023
@mjonss mjonss mentioned this pull request Jan 25, 2023
Closed
13 tasks
@asddongmen
Copy link
Contributor

Could you please fix the failed unit tests and add a ALTER TABLE ... REORGANIZE PARTITION SQL to here: https://github.com/pingcap/tiflow/blob/master/tests/integration_tests/partition_table/data/prepare.sql#L28 to test whether TiCDC can handle this DDL expectedly.

@mjonss mjonss mentioned this pull request Jan 31, 2023
Closed
12 tasks
@mjonss
Copy link
Contributor Author

mjonss commented Feb 14, 2023

Could you please fix the failed unit tests and add a ALTER TABLE ... REORGANIZE PARTITION SQL to here: https://github.com/pingcap/tiflow/blob/master/tests/integration_tests/partition_table/data/prepare.sql#L28 to test whether TiCDC can handle this DDL expectedly.

Hi @asddongmen, I have added such test as well. Could you please review it now?

@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 18, 2023
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 7, 2023
@mjonss
Copy link
Contributor Author

mjonss commented Mar 7, 2023

@asddongmen @zhaoxinyu @tangenta PTAL, I have updated the patch and included the requested test.

asddongmen
asddongmen reviewed Mar 16, 2023
View reviewed changes
tests/integration_tests/partition_table/data/prepare.sql
@@ -26,4 +26,7 @@ ALTER TABLE t1 EXCHANGE PARTITION p3 WITH TABLE t2;
insert into t2 values (100),(101),(102),(103),(104),(105); /*these values will be replicated to in downstream t2*/
insert into t1 values (25),(29); /*these values will be replicated to in downstream t1.p3*/

ALTER TABLE t1 REORGANIZE PARTITION p0,p2 INTO (PARTITION p0 VALUES LESS THAN (5), PARTITION p1 VALUES LESS THAN (10), PARTITION p2 VALUES LESS THAN (21));
ALTER TABLE t1 REORGANIZE PARTITION p2,p3,p4 INTO (PARTITION p2 VALUES LESS THAN (20), PARTITION p3 VALUES LESS THAN (26), PARTITION p4 VALUES LESS THAN (35), PARTITION pMax VALUES LESS THAN (MAXVALUE));

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add some insert querys below line 29 and line 30 to examine TiCDC can replicate data correctly after these DDL. thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asddongmen I have added some dml after each of the two DML statements.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you ~! LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 16, 2023
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 16, 2023
@asddongmen
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: bd30743

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 16, 2023
@ti-chi-bot
Copy link
Member

@mjonss: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

trigger some heavy tests which will not run always when PR updated.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@bb7133
Copy link
Member

bb7133 commented Mar 16, 2023

/retest

@mjonss
Copy link
Contributor Author

mjonss commented Mar 16, 2023

/run-verify

@mjonss
Copy link
Contributor Author

mjonss commented Mar 16, 2023

/run-engine-integration-test

@mjonss
Copy link
Contributor Author

mjonss commented Mar 16, 2023

/run-kafka-integration-test

@mjonss
Copy link
Contributor Author

mjonss commented Mar 16, 2023

/run-MySQL-integration

@asddongmen
Copy link
Contributor

/run-verify
/run-engine-integration-tests

@ti-chi-bot ti-chi-bot merged commit 129d619 into pingcap:master Mar 16, 2023
@mjonss mjonss deleted the cdc-reorganize-partition branch March 16, 2023 23:06
Rustin170506 added a commit to Rustin170506/tiflow that referenced this pull request Mar 21, 2023
@Rustin170506
Revert "Support ALTER TABLE t REORGANIZE PARTITION (pingcap#8114)"
93ba2a9 
This reverts commit 129d619.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhaoxinyu zhaoxinyu zhaoxinyu approved these changes

@asddongmen asddongmen asddongmen approved these changes

@tangenta tangenta Awaiting requested review from tangenta

Assignees
No one assigned
Labels
first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support ALTER TABLE REORGANIZE PARTITION
6 participants
@mjonss @ti-chi-bot @sre-bot @asddongmen @bb7133 @zhaoxinyu