Amendment for Subcoin milestone3

[liuchengxu]

In light of recent developments, it has become evident that fully syncing to the tip of the Bitcoin network and enabling new nodes to perform fast sync to the latest Bitcoin state is more challenging than initially anticipated, caused by the huge state of UTXO set (over 12GiB, much larger than any existing Substrate chains to my knowledge). We'll also discuss the fast sync challenge in the article delivery at this milestone. As a result, I would like to propose adjusting the delivery goal for this milestone.

The most significant known blocker is paritytech/polkadot-sdk#4. Other underlying issues may also contribute to the difficulty. Recent experiments have shown that fast sync from around block height 580,000 is currently infeasible, succeeding only on machines with 128GiB of memory (paritytech/polkadot-sdk#5053 (comment)), which is impractical for most users. Nevertheless, we have successfully demonstrated that decentralized fast sync is possible within a prototype implementation.

While syncing to the Bitcoin network's tip remains a future target, addressing the existing technical challenges will require substantial R&D efforts. We remain committed to exploring potential solutions, including architectural changes and contributing to resolving issue paritytech/polkadot-sdk#4,

[PieWol]

Hey @liuchengxu ,
thanks for reaching out about this issue. Simply out of interest, have you tried to allow for a huge swap file so that this might be successful even on machines without 128gb of RAM?

[liuchengxu]

have you tried to allow for a huge swap file so that this might be successful even on machines without 128gb of RAM?

This is interesting. After increasing the swap to 128GiB locally, I successfully completed a fast sync from a node at height 612,272. The peak memory usage was around 128GiB of RAM plus 50GiB of swap, importing the state took about 50 minutes on my machine. However, after importing the synced state, the node took an unreasonably long time to restart. It appears that Substrate encounters significant challenges when handling state at this scale.

To conclude, a fast sync from height ~580,000 might be still possible with 128GiB of RAM and just 2GiB of swap. But when syncing from height 612,272, the memory consumption rose to 128GiB of RAM plus 50GiB of swap, which is unsustainable considering there are still over 240,000 blocks left to sync. Extrapolating from these numbers, syncing the latest Bitcoin state might require 128GiB RAM plus swap ~400GiB, which is simply unmanageable.

[Noc2]

Happy to go ahead with this.

[PieWol]

Thank you for swiftly conducting the experiment and laying out the results. The change to the grant agreement sounds very reasonable to me.