Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: fix GC of subsumed replicas #31988

Merged
merged 2 commits into from
Oct 30, 2018
Merged

Conversation

benesch
Copy link
Contributor

@benesch benesch commented Oct 29, 2018

When a range is subsumed, there is a chance that it leaves behind
replicas that are marked as destroyed with reason "merge pending." The
replica GC queue was previously misinterpreting this to mean that the
range had already been GC'd and thus the range would never get GC'd,
resulting in a proliferation of intersecting snapshot errors.

Additionally take the opportunity to teach the replica GC queue to
proactively queue a replica's left neighbor when that left neighbor is
blocking the replica from being GC'd.

Release note: None

@benesch benesch requested review from bdarnell, tbg and a team October 29, 2018 22:32
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Contributor

@bdarnell bdarnell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: but I might split the two non-test changes into separate commits (so that we have separate bisection targets in case one of them has unexpected consequences)

Reviewed 3 of 3 files at r1.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale)

@benesch
Copy link
Contributor Author

benesch commented Oct 29, 2018

but I might split the two non-test changes into separate commits (so that we have separate bisection targets in case one of them has unexpected consequences)

Ack, will do after this CI cycle.

When considering a subsumed replica R for GC, we need to prove that its
left neighbor is generationally up to date. If we notice that this left
neighbor is not generationally up to date, it is likely that it needs a
GC run itself, so queue it.

Release note: None
When a range is subsumed, there is a chance that it leaves behind
replicas that are marked as destroyed with reason "merge pending." The
replica GC queue was previously misinterpreting this to mean that the
range had already been GC'd and thus the range would never get GC'd,
resulting in a proliferation of intersecting snapshot errors.

Release note: None
@benesch
Copy link
Contributor Author

benesch commented Oct 30, 2018

Ok, this is split into two separate commits. The test ended up being flaky (because of course it did). I'm going to merge with the flaky test skipped and fix and unskip it in a future commit so that this lands in time for the nightly roachtest run.

bors r=bdarnell

craig bot pushed a commit that referenced this pull request Oct 30, 2018
31988: storage: fix GC of subsumed replicas r=bdarnell a=benesch

When a range is subsumed, there is a chance that it leaves behind
replicas that are marked as destroyed with reason "merge pending." The
replica GC queue was previously misinterpreting this to mean that the
range had already been GC'd and thus the range would never get GC'd,
resulting in a proliferation of intersecting snapshot errors.

Additionally take the opportunity to teach the replica GC queue to
proactively queue a replica's left neighbor when that left neighbor is
blocking the replica from being GC'd.

Release note: None

Co-authored-by: Nikhil Benesch <nikhil.benesch@gmail.com>
@craig
Copy link
Contributor

craig bot commented Oct 30, 2018

Build succeeded

@craig craig bot merged commit 3cc9d57 into cockroachdb:master Oct 30, 2018
@benesch benesch deleted the merge-gc branch October 30, 2018 01:50
Copy link
Member

@tbg tbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r2, 2 of 2 files at r3.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants