Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rust parse serialized #3444

Merged
merged 2 commits into from
May 3, 2021
Merged

Rust parse serialized #3444

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c13470c
use rust implementation for finding length of a serialized clvm program
arvidn May 2, 2021
4b0be19
bump clvm_rs version
arvidn May 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
use rust implementation for finding length of a serialized clvm program
  • Loading branch information
@arvidn
arvidn committed May 2, 2021
commit c13470c4f379d586034bacdddf6351014f995b0d
8 changes: 4 additions & 4 deletions chia/types/blockchain_format/program.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
from clvm.casts import int_from_bytes
from clvm.EvalError import EvalError
from clvm.operators import OP_REWRITE, OPERATOR_LOOKUP
from clvm.serialize import sexp_buffer_from_stream, sexp_from_stream, sexp_to_stream
from clvm_rs import STRICT_MODE, deserialize_and_run_program
from clvm.serialize import sexp_from_stream, sexp_to_stream
from clvm_rs import STRICT_MODE, deserialize_and_run_program, serialized_length
from clvm_tools.curry import curry, uncurry

from chia.types.blockchain_format.sized_bytes import bytes32
Expand Down Expand Up @@ -148,8 +148,8 @@ class SerializedProgram:

@classmethod
def parse(cls, f) -> "SerializedProgram":
tmp = sexp_buffer_from_stream(f)
return SerializedProgram.from_bytes(tmp)
length = serialized_length(f.getvalue()[f.tell() :])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible that casting the bytes into a memoryview will save one copy of the blob. Might be worth checking to see how it benchmarks.

length = serialized_length(memoryview(f.getvalue())[f.tell():])

(I've been meaning to experiment with memoryview in python clvm to see if it makes for zero-cost substr.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, my initial idea was to use getbuffer(), but to feel confident with that API I would want to add more tests to the actual python bindings of clvm_rs too.

I agree that a longer term solution would be to take buffer views.

return SerializedProgram.from_bytes(f.read(length))

def stream(self, f):
f.write(self._buf)
Expand Down