Map the entire Ethereum State into IPFS with ipld

[wanderer]

It would be super cool if we could map the entire Ethereum state into ipfs with the help of IPLD.
Here is the proposed layout.

  eth\-block hash-\
                   \-previous block
                   |
                   |-txs-\
                   |     |-tx 0
                   |     |-tx 1
                   |     \-tx N
                   |
                   |-uncles-\
                   |        |- uncle 0
                   |        |- uncle 1
                   |        \ -uncle N
                   |
                   |-header-\
                   |        |-number
                   |        |-mix
                   |        |-nonce
                   |        |-difficulty
                   |        |-bloom
                   |        |-coinbase
                   |        |-gasLimit
                   |        |-timestamp
                   |        \-extraData
                   |
                   \--accounts-\
                                \--address N--\
                                               \--balance
                                               |--nonce
                                               |--code
                                               \--storage--\
                                                            \-key 0
                                                            |-key 1
                                                            |-key N

some initial work has begun by adding RLP to multicodec

I think the next step is to be able to serialize the ethereum merkle patricia nodes correctly. Where do we start? Where exactly can we plug in the logic for to serialize and unserialize ethereum merkle patricia nodes? :)

[wanderer]

related multiformats/multicodec#16

[wanderer]

also needed ipld/js-ipld#15

[daviddias]

Created a tracking issue to aggregate all the endeavours (and to keep everyone updated) ipfs/notes#173 :)

[kumavis]

In order for pathing to work nicely ( and maintain cononical hashes ) for all ethereum objects, I think we will need several ethereum codecs

my list so far:

• eth-block (header only)
• eth-block-list (uncles)
• eth-tx-trie
• eth-tx-receipt-trie
• eth-tx
• eth-state-trie
• eth-account

ethereum full blocks (bodies) wont exist in this model, just block headers and separate lookups for the uncles, txs, txReceipts

it would be nice if we could just have an rlp single-codec solution, but that doesn't tell ipld what the sub content is or how to resolve paths against it

heres an example in cid pseudocode:
with eth-block, eth-tx-trie, and eth-tx codec:

@hex.cidv2.eth-block.keccak_256.{blockHash}/transactions/0/value

with rlp and eth-patricia codecs:

@hex.cidv2.rlp.keccak_256.{blockHash}/4
@hex.cidv2.eth-patricia.keccak_256.{txTrieRoot}/0
@hex.cidv2.rlp.keccak_256.{txHash}/4

[kumavis]

I started my js-ipld-eth-block work here ipld/js-ipld-eth-block#2
still rather early but the parts are there

[Stebalien]

That still leaves the question of whether or not we want to maintain these hashes: #29 (comment)

[daviddias]

@Stebalien we need to issue a comment clarifying it better, but tl;dr; is that with CID and IPLD, you won't need to have 'IPLD-CBOR' hashes, the ethereum native data structs will ride IPFS just fine, this means, you don't have to touch those RLP encoded blocks.

[Stebalien]

@diasdavid please see my comment. My point is that adding a new format (CID) for every new system we come across will drastically increase the complexity of implementing a feature-complete IPLD library for little to no gain.

[jbenet]

@kumavis is it possible to bundle those ethereum codecs into one? is it possible for one codec to discern between those? For example, in git, the object itself discerns between {commit, blob, tree}, and thus a single codec can mean git-object, and implement the proper tree, resolve functions for each of the types internally.

Basically, one git IPLD codec implements support for all three commit, blob, tree. Can we do the same with eth objects?

[kumavis]

@jbenet the content is not self-identifying so "no", but we may be able to use some clever heuristics. will investigate.

[wanderer]

@kumavis is it possible to bundle those ethereum codecs into one?

@kumavis @jbenet I think there is a way. All ethereum objects are encoded as RPL arrays

• All tx's have 9 items
• All merkle patricia nodes have 17 items
• All blocks have have 3 items
• All accounts have 4 items
• All receipt 3 items

eth-tx-trie, eth-tx-receipt-trie, eth-state-trie, are all the same thing, they just contain different values at the leaf nodes

[kumavis]

@wanderer for {eth-tx-trie, eth-tx-receipt-trie, eth-state-trie}, if we look at the leaves we can prolly guess if they are {tx, txReceipt, or binary data}. Could the binary data give us a false positive for {tx, txReceipt}? or will the length make it clear?

[kumavis]

one point of difference between {eth-tx-trie, eth-tx-receipt-trie} and {eth-state-trie} is that the former does not hash its keys, and the latter does. I wonder what the best way to handle this is...

[wanderer]

tx, txReceipt, or binary data

its all binary data. after we RPL.decode, then we can count the array elements and know what type it is.

{eth-tx-trie, eth-tx-receipt-trie} and {eth-state-trie} is that the former does not hash its keys, and the latter does.

right, but it shouldn't matter from IPFS view point. It just need a path. The client when it needs to looks up and account will have to hash the address which will then be that path that IPFS resolves

[kumavis]

The client when it needs to looks up and account will have to hash the address which will then be that path that IPFS resolves

really ruins human readability of the path. the cost of preserving that is having a couple extra ethereum object types. I think the benefit may outweigh the cost.

[rvagg]

Closing due to staleness as per team agreement to clean up the issue tracker a bit (ipld/team-mgmt#28). This doesn't mean this issue is off the table entirely, it's just not on the current active stack but may be revisited in the near future. If you feel there is something pertinent here, please speak up, reopen, or open a new issue. [/boilerplate]