Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TrieTree implementation #34

Merged
merged 25 commits into from
Nov 21, 2022
Merged

TrieTree implementation #34

merged 25 commits into from
Nov 21, 2022

Conversation

code-monad
Copy link
Contributor

@code-monad code-monad commented Nov 10, 2022

This PR brings a new implementation for SparseMerkleTree (check trie_tree.rs), typically called TrieTree, which is:

  • Less read/write operations while updating compared to the original one
  • Less storage space compared to the original one
  • Compatible with the original tree hashes
  • Same root if tree has same leaf
  • Same MerkleProof&Compiled MerkleProof
  • *The branches_map won't be the same since the storage is not the same

Reference

Read/Write performace comparison:

Here's some bench data with rw operations counters
This can reproduced by the benchmark : benches/store_counter_benchmark.rs
with command: cargo bench --features trie -- --test and cargo bench -- --test

New

cargo bench --features trie -- --test
#...
random update 100 keys, store counters: Counters { get_branch_counter: 1740, get_leaf_counter: 0, insert_branch_counter: 660, insert_leaf_counter: 100, remove_branch_counter: 0, remove_leaf_counter: 0 }
random update 10000 keys, store counters: Counters { get_branch_counter: 370570, get_leaf_counter: 0, insert_branch_counter: 131628, insert_leaf_counter: 10000, remove_branch_counter: 0, remove_leaf_counter: 0 }
random update_all 100 keys, store counters: Counters { get_branch_counter: 1731, get_leaf_counter: 0, insert_branch_counter: 657, insert_leaf_counter: 100, remove_branch_counter: 0, remove_leaf_counter: 0 }
random update_all 10000 keys, store counters: Counters { get_branch_counter: 371504, get_leaf_counter: 0, insert_branch_counter: 131988, insert_leaf_counter: 10000, remove_branch_counter: 0, remove_leaf_counter: 0 }

Original

cargo bench -- --test
#...
random update 100 keys, store counters: Counters { get_branch_counter: 25600, get_leaf_counter: 0, insert_branch_counter: 25600, insert_leaf_counter: 100, remove_branch_counter: 0, remove_leaf_counter: 0 }
random update 10000 keys, store counters: Counters { get_branch_counter: 2560000, get_leaf_counter: 0, insert_branch_counter: 2560000, insert_leaf_counter: 10000, remove_branch_counter: 0, remove_leaf_counter: 0 }
random update_all 100 keys, store counters: Counters { get_branch_counter: 24836, get_leaf_counter: 0, insert_branch_counter: 24935, insert_leaf_counter: 100, remove_branch_counter: 0, remove_leaf_counter: 0 }
random update_all 10000 keys, store counters: Counters { get_branch_counter: 2418267, get_leaf_counter: 0, insert_branch_counter: 2428266, insert_leaf_counter: 10000, remove_branch_counter: 0, remove_leaf_counter: 0 }

Code Monad and others added 9 commits November 10, 2022 17:01
* TrieTree -- A compacted SparseMerkleTree
* Removed unused code, fix warnings, code path tweaking
* Merkle Proof implementation(inclusion&exclusion), compatible with original one
@quake quake requested review from TheWaWaR, jjyr and quake and removed request for TheWaWaR and jjyr November 10, 2022 09:24
src/lib.rs Show resolved Hide resolved
src/merge.rs Show resolved Hide resolved
src/merge.rs Outdated Show resolved Hide resolved
src/merge.rs Outdated Show resolved Hide resolved
src/merge.rs Outdated Show resolved Hide resolved
src/merge.rs Outdated Show resolved Hide resolved
src/merge.rs Outdated Show resolved Hide resolved
1. use get_bit for a more clearfy operation
2. ignore height to make less confusion

Co-authored-by: LinFeng <thewawar@gmail.com>
src/merge.rs Show resolved Hide resolved
src/trie_tree.rs Outdated Show resolved Hide resolved
src/trie_tree.rs Outdated Show resolved Hide resolved
src/trie_tree.rs Outdated Show resolved Hide resolved
Co-authored-by: Quake Wang <quake.wang@gmail.com>
@code-monad code-monad requested review from quake and removed request for jjyr November 14, 2022 01:02
@quake quake requested a review from jjyr November 14, 2022 01:08
src/trie_tree.rs Outdated Show resolved Hide resolved
src/trie_tree.rs Outdated Show resolved Hide resolved
src/trie_tree.rs Outdated Show resolved Hide resolved
@TheWaWaR
Copy link
Collaborator

Others just look good to me, I'll approve if all above comments resolved.

Copy link
Collaborator

@TheWaWaR TheWaWaR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add test case for zero-value shortcut issue

@code-monad code-monad requested review from TheWaWaR and removed request for jjyr November 17, 2022 02:22
Code Monad and others added 4 commits November 18, 2022 16:02
…th's height == 0, MergeValue::shortcut should be MergeValue::Value
* refactor: improve shortcut node builds logic and into_merge_value

* Apply suggestions from code review

Co-authored-by: Quake Wang <quake.wang@gmail.com>

* chore: fix build fail

Co-authored-by: Quake Wang <quake.wang@gmail.com>
@quake quake merged commit 2641f59 into nervosnetwork:master Nov 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants