Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

txn: fix pessimitic exist check #19004

Merged
merged 2 commits into from
Aug 6, 2020
Merged

txn: fix pessimitic exist check #19004

merged 2 commits into from
Aug 6, 2020

Conversation

cfzjywxk
Copy link
Contributor

@cfzjywxk cfzjywxk commented Aug 5, 2020 •
edited
Loading

What problem does this PR solve?

Issue Number: close #18958

Problem Summary:

The pessimistic lock added by the previous SQL statements will invalid the latter existence check on some keys, some duplicates are not checked and the transaction commit successfully which should fail.

What is changed and how it works?

What's Changed:

  • Record the keys having presume not exist flag in statement context.
  • Record pessimistic lock values for both point get and non point get locks, these value will be used to check existence.
  • Check existence if needed in LockKeys function for pessimistic transactions, report error if the value already exists.

How it Works:

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test

Side effects

Release note

  • fix the existence checks for pessimistic transactions.

@cfzjywxk cfzjywxk added type/bugfix This PR fixes a bug. require-LGT3 Indicates that the PR requires three LGTM. sig/transaction SIG:Transaction labels Aug 5, 2020
@cfzjywxk cfzjywxk requested a review from a team as a code owner August 5, 2020 07:57
@cfzjywxk cfzjywxk requested review from XuHuaiyu and removed request for a team August 5, 2020 07:57
@coocood
Copy link
Member

coocood commented Aug 5, 2020

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 5, 2020
bobotu
bobotu reviewed Aug 5, 2020
View reviewed changes
Copy link
Contributor

@bobotu bobotu left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this will conflict with #18966.
BTW, CheckKeyExists can be a KeyFlags to reduce memory allocation & usage.

This patchset will cherry-pick to release branch, KeyFlags only exists on the master. I'll resolve conflict in #18966 after this PR merged.

bobotu
bobotu reviewed Aug 5, 2020
View reviewed changes
Copy link
Contributor

@bobotu bobotu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 5, 2020
@cfzjywxk
Copy link
Contributor Author

cfzjywxk commented Aug 5, 2020

Seems like this will conflict with #18966.

BTW, CheckKeyExists can be a KeyFlags to reduce memory allocation & uasge.

The key flag for CheckKeyExists will be reset for each statement, which keyFlag interface should I use to do such reset? I think I need to spend some time to learn about the new interfaces first 🤔.

@bobotu
Copy link
Contributor

bobotu commented Aug 5, 2020

Seems like this will conflict with #18966.
BTW, CheckKeyExists can be a KeyFlags to reduce memory allocation & uasge.

The key flag for CheckKeyExists will be reset for each statement, which keyFlag interface should I use to do such reset? I think I need to spend some time to learn about the new interfaces first 🤔.

KeyFlags don't support reset for each statement🤣.

Due to we need cherry-pick this commit to release branch which uses the older memdb, we'd better not use any new memdb features in this PR.

lysu
lysu previously approved these changes Aug 5, 2020
View reviewed changes
Copy link
Contributor

@lysu lysu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT3 The PR has already had 3 LGTM. and removed status/LGT2 Indicates that a PR has LGTM 2. labels Aug 5, 2020
ti-srebot
ti-srebot previously approved these changes Aug 5, 2020
View reviewed changes
@lysu
Copy link
Contributor

lysu commented Aug 5, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 5, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@cfzjywxk merge failed.

@codecov
Copy link

codecov bot commented Aug 5, 2020 •
edited
Loading

Codecov Report

Merging #19004 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #19004   +/-   ##
===========================================
  Coverage   79.4730%   79.4730%           
===========================================
  Files           546        546           
  Lines        148522     148522           
===========================================
  Hits         118035     118035           
  Misses        20996      20996           
  Partials       9491       9491

@bobotu
Copy link
Contributor

bobotu commented Aug 5, 2020

/merge

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@cfzjywxk merge failed.

@cfzjywxk
Copy link
Contributor Author

cfzjywxk commented Aug 5, 2020 •
edited
Loading

@cfzjywxk merge failed.

Seems something wrong that test session package with tikv will get stuck.
The TestPessimisticReadCommitted case.

@cfzjywxk cfzjywxk mentioned this pull request Aug 5, 2020
Merged
@bobotu
Copy link
Contributor

bobotu commented Aug 5, 2020

/run-all-tests

@cfzjywxk
Copy link
Contributor Author

cfzjywxk commented Aug 6, 2020

@cfzjywxk merge failed.

Seems something wrong that test session package with tikv will get stuck.
The TestPessimisticReadCommitted case.

The failure is because if pessimistic write conflict happens in handlePessimisticDML, the checkKeysExists map will not be reset, the case will fail and get stuck.

ti-srebot pushed a commit that referenced this pull request Aug 6, 2020
@cfzjywxk
txn: fix pessimitic exist check(#19004) (#19019)
4fb518b 
* txn: fix pessimitic exist check

* remove log
@cfzjywxk
Copy link
Contributor Author

cfzjywxk commented Aug 6, 2020

/run-all-tests

@cfzjywxk
Copy link
Contributor Author

cfzjywxk commented Aug 6, 2020

/merge

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@cfzjywxk merge failed.

@bobotu
Copy link
Contributor

bobotu commented Aug 6, 2020

/run-integration-copr-test

@cfzjywxk
Copy link
Contributor Author

cfzjywxk commented Aug 6, 2020

/merge

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@cfzjywxk merge failed.

@cfzjywxk
Copy link
Contributor Author

cfzjywxk commented Aug 6, 2020

/merge

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@cfzjywxk merge failed.

@bobotu
Copy link
Contributor

bobotu commented Aug 6, 2020

/run-unit-test

@bobotu
Copy link
Contributor

bobotu commented Aug 6, 2020

/merge

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@cfzjywxk merge failed.

@bobotu
Copy link
Contributor

bobotu commented Aug 6, 2020

/run-check_dev_2

@coocood coocood merged commit 574540a into pingcap:master Aug 6, 2020
@cfzjywxk cfzjywxk mentioned this pull request Aug 17, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bobotu bobotu bobotu left review comments

@coocood coocood coocood approved these changes

@lysu lysu lysu left review comments

@ti-srebot ti-srebot ti-srebot left review comments

@jackysp jackysp Awaiting requested review from jackysp

@sticnarf sticnarf Awaiting requested review from sticnarf

@XuHuaiyu XuHuaiyu Awaiting requested review from XuHuaiyu

Assignees
No one assigned
Labels
component/executor require-LGT3 Indicates that the PR requires three LGTM. sig/transaction SIG:Transaction status/can-merge Indicates a PR has been approved by a committer. status/LGT3 The PR has already had 3 LGTM. type/bugfix This PR fixes a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

An insert statement which violate key constraint can be committed successfully
5 participants
@cfzjywxk @coocood @bobotu @lysu @ti-srebot