Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wasm Feature Branch #959

Draft
wants to merge 108 commits into
base: main
Choose a base branch
from
Draft

Wasm Feature Branch #959

wants to merge 108 commits into from

Conversation

insipx
Copy link
Contributor

@insipx insipx commented Aug 14, 2024

Tracking #942

closes #940 #941 #1038 #1034 #1104

  • refactors EncryptedMessageStore into traits & exposes a type alias depending on compile target
  • refactors StreamHandles into traits & exposes type alias depending on compile target
  • adds CI-adjacent cargo clippy to ./dev/lint-rust script
  • All crates apart of same workspace. should speed up CI
  • Introduce trait, ScopedGroupClient held on MlsGroup
  • Add Client functionality back to MlsGroup
    • MlsGroup accesses client methods through ScopedGroupClient. This keeps
      the distinction between MlsGroup and Client clear (while also erasing type parameters), while opening it up to
      future refactors where we pull MlsGroup further away from client. (The subset of functions MlsGroup relies on from Client is actually rather small src/groups/scoped_client.rs)

Much of this PR is changing attributes to account for webassembly (tokio-test vs wasm_bindgen::test, etc.)

Some more interesting bits to pay attention to:

  • encrypted_store/mod.rs, encrypted_store/native.rs, encrypted_store/wasm.rs, and stream_handles.rs.
    • There's probably a better way to organize encrypted_store/mod.rs, however the current way keeps the details well away from the Client or any code using the store. From the outside, it looks like nothing really changed.

the patch dependency for openssl-sys was never being applied, (latest kotlin build for affected target):

warning: Patch `openssl-sys v0.9.92 (https://github.com/xmtp/rust-openssl.git?branch=clone-v0.9.92#00dacb93)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.

We have an openssl dependency for sqlite. after bringing all crates into the workspace, I was able to pin it:

cargo tree -i openssl-sys
openssl-sys v0.9.92
├── libsqlite3-sys v0.29.0
│   ├── diesel v2.2.0 (https://github.com/xmtp/diesel?branch=insipx/sqlite-replace-ignore-pub#78ece4f8)
│   │   ├── diesel_migrations v2.2.0 (https://github.com/xmtp/diesel?branch=insipx/sqlite-replace-ignore-pub#78ece4f8)
│   │   │   └── xmtp_mls v0.1.0 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/xmtp_mls)
│   │   │       ├── bindings_node v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/bindings_node)
│   │   │       ├── bindings_wasm v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/bindings_wasm)
│   │   │       ├── mls_validation_service v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/mls_validation_service)
│   │   │       ├── xmtp_cli v0.1.0 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/examples/cli)
│   │   │       └── xmtpv3 v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/bindings_ffi)
│   │   │       [dev-dependencies]
│   │   │       └── xmtpv3 v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/bindings_ffi)
│   │   └── xmtp_mls v0.1.0 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/xmtp_mls) (*)
│   └── xmtp_mls v0.1.0 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/xmtp_mls) (*)
├── openssl v0.10.57
│   └── xmtp_mls v0.1.0 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/xmtp_mls) (*)
└── xmtp_mls v0.1.0 (/Users/insipx/Projects/xmtp/workspace-libxmtp/wasm-backend/xmtp_mls) (*)

vs main:

❯ cargo tree -i openssl-sys
warning: Patch `openssl-sys v0.9.92 (https://github.com/xmtp/rust-openssl.git?branch=clone-v0.9.92#00dacb93)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.
openssl-sys v0.9.102
└── libsqlite3-sys v0.29.0
    ├── diesel v2.2.2
    │   ├── diesel_migrations v2.2.0
    │   │   └── xmtp_mls v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/main/xmtp_mls)
    │   │       └── xmtpv3 v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/main/bindings_ffi)
    │   │       [dev-dependencies]
    │   │       └── xmtpv3 v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/main/bindings_ffi)
    │   └── xmtp_mls v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/main/xmtp_mls) (*)
    └── xmtp_mls v0.0.1 (/Users/insipx/Projects/xmtp/workspace-libxmtp/main/xmtp_mls) (*)

Functions Ported

Co-authored-by: Ry Racherbaumer <ry@xmtp.com>
@insipx insipx requested review from fabriguespe and a team as code owners August 14, 2024 17:17
@insipx insipx marked this pull request as draft August 14, 2024 17:17
insipx and others added 5 commits August 15, 2024 11:35
* statement stream

* fix lifetimes on load query

* execute returning count

* transaction manager

* cleanup
* fix diesel function registration

* query building works

* try insert with orml

* compile with sqlite default fix

* compile but still ahve static lifetimes

* fix unimplemented traits

* ORM compiles!

* some cleanup, setup tracing in wasm for test

* readme

* upgrade wa-sqlite version

* temp DebugQuery to help with debugging

* make debug query unsafe

* progress on statement issue

* progress for destructor order

* logs/fix async drop for BoundStatement

* remove task spawn for statementfactory
* use official sqlite bindings

* PORTED

* can establish conn

* getters/setters

* constant from JS workaround

* r2d2 connection

* add custom allocator for faster memory

* memory optimization, tests

* clean up dependencies

* clean

* workflow

* readme

* dep pruning

* readme change

* readme
@insipx insipx force-pushed the wasm-backend branch 4 times, most recently from 2187e91 to 68132b2 Compare August 26, 2024 22:14
* organize wasm tests so they all run

* make test organization much better, fix tests

* fix workflow file
insipx and others added 15 commits August 27, 2024 15:35
remove rollup plugin typescript

organize JS into one folder
* fix bind blob

* try to repro last compiler error

* UpdateAndFetchResults

* remove unused

* workaround for insert_with_default

* clippy
* compile xmtp_id and xmtp_mls to wasm32

* remove `native` features

* remove unsafe & dup trait

* annotate all tests with wasm-bindgen-test for wasm32

* xmtp_mls tests compile for wasm32

* dont create a new file for database in wasm32

* fix compile for bindings_wasm

* pin to wasm-bindgen 0.2.92

* fix wasm-bindgen-test version
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Include diesel-wasm-sqlite crate in xmtp_mls
4 participants