[2] Waku2 Usage for Community History

[staheri14]

Problem

How to use the current capabilities of waku2 to address the community history issue.

Solution idea

In the following solution, we follow the modular approach introduced in #82 and describe how each module can be implemented and realized through the current capabilities of waku2.

Current Waku2 capabilities

Message ordering mechanism

• sender generated timestamp + message hash (this allows total message ordering)
• Works based on the assumption that nodes have synchronized clocks and with the maximum of +-20 seconds asynchrony

Access control management/ Group management

• Out of band
• List of the group members with their keys should be communicated out of band.
• Can enforce read access by using encryption. Waku message payload is encrypted and only users with a decryption key can decipher the message.
• Can enforce write access by using signature. Each encrypted message has the encrypted sender signature as well and this can be verified by the receiving entity to whether trust the sender or not.
• The access control list is not visible to the store nodes (unless they are also part of the community and among the authorized nodes). Due to this limitation, they won't be able to grant or deny read and write access to the message history hence they will accept all the reads and write requests.

State synchronization/consensus
The current state synchronization of the Waku2 store protocol is empowered by the Waku FT-store protocol.

• Store nodes (while being online) listen to the network and store all the messages of the community.
• Store nodes are allowed to go offline and then recover the gap in their message history by performing time-based queries from another store node that has been online for that duration.
• Waku2 FT-store works based on the following assumptions
  • At any time, there is at least one online store node from which the history can be retrieved
  • Nodes (including store nodes and other regular nodes who publish messages) have synchronized clocks with a +-20 seconds error margin
  • There is no network partition, that is all the store nodes will ultimately receive all the messages transmitted in the network
• Under the aforementioned assumptions, FT-store provides eventual consistency. However, the message state of store nodes may diverge for some points in time due to network delay.

Discovery

• Discovery is static and out of band / centralized
• Consensus nodes should be configured with a static set of other store nodes

Transport

• The current transport layer of Waku2 implements libp2p Gossipsub which is a pubsub messaging system

Offline/ long-term storage

• The store nodes are supposed to have enough storage to maintain the entire history

Resource Throttling
Not supported

A minimal solution based on the current Waku2 capabilities

Below is how can we address community history leveraging the current capabilities of Waku2.

The community owner specifies a set of store nodes with their addresses and makes the addresses public for the community members to see.

At least one store node should be online for each given time window.

Store nodes are supposed to be trusted so that they don't hide any message and behave as instructed.

All the store nodes are participants of the transport layer i.e., Gossipsub.

The community owner runs the store nodes and configures them with the static list of store nodes. As the result, it creates a full graph between the store nodes.

Store nodes are supposed to have synchronized clocks with at most +-20 sec diffs.

Store nodes begin listening to and storing all the messages published in the network through the transport layer.

If a store node goes offline, once gets back online, it queries all the other store nodes for that offline time window. It then fixes its history gap.

Each store node can reliably provide message history to the community members.

Each store node can order messages based on their timestamp + hash. It provides total ordering.

In this design, there is no support for the update or deletion of messages. Though, it should be easy to incorporate further fields in the waku message to indicate an update/delete request for a previously published message.

Ongoing and future work

Message ordering mechanism: #76
Access control management/ Group management: #79
Store synchronization/Consensus: #80
Discovery: #81
Offline/long-term storage: #78
Resource throttling: #84
Edit/Remove messages: #77

[oskarth]

@staheri14 @PascalPrecht can we link this issue with existing work in this area please? I know work continued in separate issues but there's no backlink to this so it is hard to follow trail.

[0x-r4bbit]

@oskarth Sure!

So far the only additional "work" with regards to specification was done here in the feature specs repo for Status Deskop: status-im/feature-specs#36

This is just a draft though with a high level overview and "expectations" from a desktop's user's perspective.
It also cross-links all the existing issues @staheri14 has created related to this problem.

Implementation hasn't really started yet as we're still figuring out which MVP approach to settle on.