Tracking: WASM Bindings (Group Chat Prod - Web)

[nplasterer]

Must Have

• DB Storage layer (LARGE) Tracking: MLS WASM DB/Storage (Group Chat Prod - Web) #556 Tracking: MLS/WASM DB Storage #942
• API the xmtp_mls client needs to be adapted to accept an abstract XmtpApiClient & create a WASM-friendly implementation in xmtp_api_grpc_gateway #563
• bindings_wasm that mirror bindings_ffi https://github.com/rustwasm/wasm-bindgen (XXL)
  • Client
    • create_client #565 *Node
    • can_message #566 *Node
    • text_to_sign #567 *Node
    • register_identity #568 *Node
    • conversations #569 *Node
    • account_address #570
  • Conversation
    • create_group #571
    • list #572
    • sync #573 *Node
    • stream #574 *Node
    • streamAllMessages #575 *Node
  • Group
    • send #576 *Node
    • sync #577 *Node
    • find_messages #578
    • list_members #579
    • add_members #580
    • remove_members #581
    • stream #582 *Node
    • leave #583
    • is_active #584 *Node
    • group_metadata #585
    • who_added_me #586
    • message_kind #587
  • Reducing the binary size for libxmtp. We need to measure this and make sure it's not enormous, and prune/modularize/swap out dependencies if it is.

[bwcDvorak]

Github, your 'convert to item' did me dirty when it came to converting issue 564, formerly a task about making the WASM bindings with a reference to an external URL. Apologies @nplasterer - can you please edit the correct title back into that issue?

[nplasterer]

@bwcDvorak I think that item is just a header and doesn't need to be an issue. I restored it from the edit history. It was just linking out to the repo that will help with the bindings.

[bwcDvorak]

TYVM!

[neekolas]

We've talked a lot about WASM bindings in terms of "make everything happen in Rust". The two sticking points are disk access (SQLite) and networking (GRPC Gateway or GRPC Web). To access the device APIs to do this in Rust, we'd need to use WASI. WASI is very bleeding edge, and making it work in a full-featured app that supports things like async and streaming, is going to be a big lift.

We haven't spent a lot of time discussing the other alternative to make libxmtp work on the web: doing all this work in JS. WASM (without WASI) supports calling into JS functions from Rust. We already have a trait for the API layer, where we could implement it in Javascript and write a JS -> Rust binding to expose an implementation of that trait (or use the pre-existing fetch API binding that already exists as part of web-sys) to write some of the business logic in Rust on top of the fetch API. We would then need to convert our storage layer into a trait, and likely refactor a bunch of our code that deals with things like transactions to support a generic storage interface that could be implemented in JS using IndexedDB.

Risks:

• Support for streaming and passing callbacks back and forth between Rust and JS
• Complicated refactor of our storage-related code to be more generic
• Is IndexedDB able to do everything we currently do with SQLite. Some functions might need to be done in a hacky way.
• Performance. This will be slower. How much slower? I don't know

I don't think we know enough yet for me to advocate for this solution. But it's a much clearer path than WASI and probably deserves a spike to derisk.

[insipx]

Wasm runtimes support Host Functions. This seems to be similar to this, where we would expose a rust trait that JS needs to implement and pass host functions into the generated WASM blob. These host functions would generally be write/read etc, whatever SQLite normally does in the rust code. Could be very close to the trait Cryspen made for OpenMLS. This would require a bit of work on the JS side, but it does circumvent the need for WASI, at least until its in a more stable state.

The downside here is we would need to convert all our sqlite insert/read code to a generic trait and implement it on the rust side too.

[neekolas]

The fact that wasm_bindgen supports really complicated browser APIs in web-sys (it can do things like use WebGL) gives me hope that the host function support is pretty mature and full-featured.