-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use pointers instead of binary encoding #965
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 6 files reviewed, 2 unresolved discussions (waiting on @ashish-goswami, @jarifibrahim, and @manishrjain)
structs.go, line 39 at r1 (raw file):
func (p valuePointer) Encode() []byte { b := make([]byte, vptrSize) *(*valuePointer)(unsafe.Pointer(&b[0])) = p
How fast is this? Any benchmarks compared to previous approach?
table/builder.go, line 312 at r1 (raw file):
} func u32SliceToBytes(u32s []uint32) []byte {
How fast do things get here? Can you benchmark this code for 2 things:
- Building a slice of []uint32 -> bytes. Diff in speed between two approaches.
- Accessing via getOffset and equivalent. Diff in speed between these two approaches.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✅ A review job has been created and sent to the PullRequest network.
@jarifibrahim you can click here to see the review status or cancel the code review job - or - cancel by adding [!pr] to the title of the pull request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 6 files reviewed, 2 unresolved discussions (waiting on @ashish-goswami and @manishrjain)
structs.go, line 39 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
How fast is this? Any benchmarks compared to previous approach?
Binary encoding takes 4.03ns while slice pointer takes 0.83ns
See https://gist.github.com/jarifibrahim/30237927ff3a4b200d4907c97bd93f41 for benchmark code
table/builder.go, line 312 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
How fast do things get here? Can you benchmark this code for 2 things:
- Building a slice of []uint32 -> bytes. Diff in speed between two approaches.
- Accessing via getOffset and equivalent. Diff in speed between these two approaches.
For 1. see https://gist.github.com/jarifibrahim/30237927ff3a4b200d4907c97bd93f41 for benchmark code
name time/op
EncodingSingle/Binary-16 4.03ns ± 1%
EncodingSingle/Slice-16 0.83ns ± 1%
Encodingu32Slice/Binary-16 9.48µs ± 1%
Encodingu32Slice/Slice-16 0.94ns ± 0%
For 2, each getOffSet
call should take 4.03ns while and converting the entire slice via pointers takes 0.94ns
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 6 files at r1, 5 of 5 files at r2.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @ashish-goswami, @jarifibrahim, and @manishrjain)
structs.go, line 39 at r1 (raw file):
Previously, jarifibrahim (Ibrahim Jarif) wrote…
Binary encoding takes 4.03ns while slice pointer takes 0.83ns
See https://gist.github.com/jarifibrahim/30237927ff3a4b200d4907c97bd93f41 for benchmark code
Add a comment here that you're copying the contents of p into b.
structs.go, line 40 at r2 (raw file):
b := make([]byte, vptrSize) *(*valuePointer)(unsafe.Pointer(&b[0])) = p return b[:]
return b
table/builder.go, line 281 at r2 (raw file):
Previously, ashish-goswami (Ashish Goswami) wrote…
We can move these functions to y package so that these can be reused.
Move the decode equivalent next to encoder func.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 2 of 9 files reviewed, 4 unresolved discussions (waiting on @ashish-goswami and @manishrjain)
structs.go, line 39 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Add a comment here that you're copying the contents of p into b.
Done.
structs.go, line 40 at r2 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
return b
Done.
table/builder.go, line 281 at r2 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Move the decode equivalent next to encoder func.
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 5 files at r2, 5 of 6 files at r3, 2 of 2 files at r4.
Dismissed @ashish-goswami and @manishrjain from 4 discussions.
Reviewable status: complete! all files reviewed, all discussions resolved
Signed-off-by: thomassong <thomassong2012@gmail.com>
Signed-off-by: thomassong <thomassong2012@gmail.com>
Signed-off-by: thomassong <thomassong2012@gmail.com>
The change improves encoding/decoding performance by 4X. See https://gist.github.com/jarifibrahim/30237927ff3a4b200d4907c97bd93f41 for benchmark code.
The change also improves the performance of existing benchmarks.
Inspired by pingcap@0b03ef0
This change is