Fix discard stats moved by GC bug (#929) #1098
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jarifibrahim
requested review from
ashish-goswami,
manishrjain and
a team
as code owners
![pullrequest[bot]](https://avatars.githubusercontent.com/in/3434?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ A review job has been created and sent to the PullRequest network.
@jarifibrahim you can click here to see the review status or cancel the code review job.
danielmai
approved these changes
Oct 24, 2019
There was a problem hiding this comment.
Reviewed 2 of 2 files at r1.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on @ashish-goswami and @manishrjain)
|
Adding #929 to release/1.6 branch. |
The discard stats are stored like normal keys in badger, which means if the size of the value of discard stats key is greater the value threshold it will be stored in the value log. When value log GC happens, we insert the same key in badger with a !badger!move prefix which points to the new value log file. So when searching for the value of a key, if the value points to a value log file which doesn't exist, we search for the same key with !badger!move prefix. The above logic was missing from the populateDiscardStats function. The earlier implementation would never search for the key with !badger!move prefix. It would return error on the first attempt to find the key. This commit ensures we search for a key with prefix badger move if the value for the existing key pointed to a value log file that no longer exists.
jarifibrahim
force-pushed
the
ibrahim/1.6-disard-stats-moved-bug
branch
from
c9212f1
to
596d5ef
Compare
There was a problem hiding this comment.
⚠️ Warning
PullRequest detected a force-push on this branch. This may have caused some information to be lost, and additional time may be required to complete review of the code. Read More
|
This branch does not contain the fixes for windows build and that's why it's failing. |
danielmai
reviewed
Oct 24, 2019
There was a problem hiding this comment.
Windows fixes can go in a separate PR. This looks good to merge on my side to fix the issue on Linux environments.
Reviewable status: 1 of 2 files reviewed, all discussions resolved (waiting on @ashish-goswami, @danielmai, and @manishrjain)
danielmai
added a commit
to dgraph-io/dgraph
that referenced
this pull request
Oct 24, 2019
This vendors in Badger v1.6.0 with dgraph-io/badger#929 cherry-picked into it (dgraph-io/badger#1098) to avoid any breaking changes to the data format.
danielmai
added a commit
to dgraph-io/dgraph
that referenced
this pull request
Oct 24, 2019
This vendors in Badger v1.6.0 with dgraph-io/badger#929 cherry-picked into it (dgraph-io/badger#1098) to avoid any breaking changes to the data format. Vendor in Badger to fix a bug that prevents Alpha from starting up successfully. This happens in v1.0.16 and v1.0.17. Error while creating badger KV posting store error: Unable to find log file. Please retry The release/v1.0 branch uses the vendor directory for dependencies. Badger was pulled in with the following command: govendor fetch github.com/dgraph-io/badger/...@efb9d9d15d7f7baa656e04933058f006c33a8d0f
danielmai
added a commit
to dgraph-io/dgraph
that referenced
this pull request
Oct 24, 2019
This vendors in Badger v1.6.0 with dgraph-io/badger#929 cherry-picked into it (dgraph-io/badger#1098) to avoid any breaking changes to the data format. Vendor in Badger to fix a bug that prevents Alpha from starting up successfully. This happens in v1.0.16 and v1.0.17. Error while creating badger KV posting store error: Unable to find log file. Please retry The release/v1.0 branch uses the vendor directory for dependencies. Badger was pulled in with the following command: govendor fetch github.com/dgraph-io/badger/...@efb9d9d15d7f7baa656e04933058f006c33a8d0f
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The discard stats are stored like normal keys in badger, which means if
the size of the value of discard stats key is greater the value threshold it
will be stored in the value log. When value log GC happens, we insert
the same key in badger with a !badger!move prefix which points to the
new value log file. So when searching for the value of a key, if the value
points to a value log file which doesn't exist, we search for the same
key with !badger!move prefix.
The above logic was missing from the populateDiscardStats function. The
earlier implementation would never search for the key with !badger!move
prefix. It would return error on the first attempt to find the key. This
commit ensures we search for a key with prefix badger move if the value
for the existing key pointed to a value log file that no longer exists.
Original PR #929
This change is