Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify the storage format of DDL jobs to support parallel DDL #18425

Closed
zimulala opened this issue Jul 7, 2020 · 1 comment
Closed

Modify the storage format of DDL jobs to support parallel DDL #18425

zimulala opened this issue Jul 7, 2020 · 1 comment
Labels
sig/sql-infra SIG: SQL Infra

Comments

@zimulala
Copy link
Contributor

zimulala commented Jul 7, 2020

Development Task

Background

It's a part of the issue "Support DDL parallel execution".
Currently, we use two queues to save the "add index" and other DDLs in storage. And we have two workers to handle these DDL jobs.
Problems of current storage format:

  • A current DDL job is stored by means of (special prefix), data key (encoded by meta key), and value. Therefore, for a certain queue (such as "add index" queue), write conflicts will occur when adding and deleting the DDL job.

Two suggestions

  • Store DDL job in an internal table

  • The queue storage mode will be modified. Meta key and data key will be directly changed into one key (stored in the mode of mDDLJobListKey+_jobID ).

Notice

  • handling dependence jobs
  • compatibility issues
@zimulala zimulala added status/TODO sig/sql-infra SIG: SQL Infra type/feature-request Categorizes issue or PR as related to a new feature. labels Jul 7, 2020
@zz-jason zz-jason removed the type/feature-request Categorizes issue or PR as related to a new feature. label Jul 9, 2020
@Defined2014
Copy link
Contributor

Seems we already supported parallel DDL, so close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/sql-infra SIG: SQL Infra
Projects
None yet
Development

No branches or pull requests

4 participants