Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(websocket): add WebSocket for WASM environments #4102

Merged
merged 80 commits into from
Oct 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
fff3b46
Add Websys Websocket transport
vincev Mar 26, 2023
f3874c7
Add rust version
vincev Mar 26, 2023
62ddf7f
Fix docs test
vincev Mar 26, 2023
3c0069b
Fix docs test
vincev Mar 26, 2023
5a86d7c
Include websys feature to full features.
vincev Mar 26, 2023
50e38e3
Fix semver error.
vincev Mar 26, 2023
25843d5
Fix semver error.
vincev Mar 26, 2023
f794aee
Don't use libp2p in dev-dependencies
vincev Mar 26, 2023
b8503e0
Add deprecation warning for wasm-ext.
vincev Mar 27, 2023
378c03f
Fix clippy error for wasm-ext deprecation
vincev Mar 27, 2023
631dc8b
Merge branch 'libp2p:master' into websys
vincev Mar 27, 2023
d5912c5
Rename transport package
vincev Apr 2, 2023
2e4922a
Remove deprecated mplex from doc test
vincev Apr 2, 2023
5534114
Return MultiaddrNotSupported for invalid multi addr.
vincev Apr 2, 2023
d50ea63
Use weak pointer in the Websocket callbacks to avoid cycle
vincev Apr 3, 2023
4603ca6
Set version to 0.1.0 for new package
vincev Apr 8, 2023
f5b7965
Fix Yamux doc comment
vincev Apr 8, 2023
487828a
Use different wakers for read and write
vincev Apr 8, 2023
c688acb
Use debug_assert for unexpected data format
vincev Apr 8, 2023
bbdd5fe
Merge branch 'master' into websys
vincev Apr 8, 2023
17e8799
Make websocket constructor error user friendly.
vincev Apr 8, 2023
0af2389
Rewrite style of cargo features
thomaseizinger Apr 10, 2023
4337fc9
Fix spelling of `web-sys`
thomaseizinger Apr 10, 2023
883552d
Misc clean-up
thomaseizinger Apr 10, 2023
b0b6de8
use early return
thomaseizinger Apr 10, 2023
27ef99d
Refactor booleans to state
thomaseizinger Apr 10, 2023
0fb9c50
Avoid unnecessary wake-ups
thomaseizinger Apr 10, 2023
c3f2abf
Properly implement `poll_close`
thomaseizinger Apr 10, 2023
37cb0cc
Reduce indentation
thomaseizinger Apr 10, 2023
a30256e
Use `Vec` instead of `VecDeque`
thomaseizinger Apr 10, 2023
4062264
Remove unused dependencies
thomaseizinger Apr 10, 2023
aad22e7
Add changelog entry
thomaseizinger Apr 10, 2023
c28b83c
Bump version
thomaseizinger Apr 10, 2023
ae393eb
Remove unnecessary `pub(crate)`
thomaseizinger Apr 10, 2023
8a5c568
Merge branch 'master' into websys
thomaseizinger Apr 10, 2023
f24468f
Add new changelog entry
thomaseizinger Apr 10, 2023
f44f546
Merge branch 'master' into websys
thomaseizinger Apr 28, 2023
0f267cf
Fixup changelog
thomaseizinger Apr 28, 2023
49c5d17
Prevent direct construction of `Transport`
thomaseizinger Apr 28, 2023
baace18
Fix formatting
thomaseizinger Apr 28, 2023
989b58f
Fix doc tests
thomaseizinger Apr 28, 2023
de4dc52
Merge branch 'master' into websys
thomaseizinger May 1, 2023
d3bfb66
Merge branch 'master' into websys
thomaseizinger May 2, 2023
9b8f071
Merge branch 'master' into websys
thomaseizinger Jun 22, 2023
73eb5ec
Rename crate
thomaseizinger Jun 22, 2023
4d3008a
Fix compile error after renaming
thomaseizinger Jun 22, 2023
a64f9de
Fix description
thomaseizinger Jun 22, 2023
bd828cd
Merge branch 'master' into websys
thomaseizinger Sep 6, 2023
f1590c0
Fix changelog and manifest
thomaseizinger Sep 6, 2023
da92c95
Rewrite websocket-websys transport
thomaseizinger Sep 6, 2023
459bf18
Allow passthrough of muxer and sec transport to WASM layer
thomaseizinger Sep 6, 2023
e066a12
Add websys websocket to interop tests
thomaseizinger Sep 6, 2023
375c890
Fix compile errors
thomaseizinger Sep 6, 2023
faec551
Compile libp2p-tls for wasm
thomaseizinger Sep 6, 2023
019a9d1
Don't compile libp2p-tls for emscripen or WASI
thomaseizinger Sep 6, 2023
27199e1
Change to `WS`
thomaseizinger Sep 6, 2023
8956262
Fix compile errors
thomaseizinger Sep 6, 2023
22fd383
Fix bug in protocol selection
thomaseizinger Sep 6, 2023
2f9a04e
Fix version of new crate
thomaseizinger Sep 6, 2023
33fed0d
Fix manifest
thomaseizinger Sep 6, 2023
fea198e
Misc fixes to make the tests pass
thomaseizinger Sep 6, 2023
99dff0f
Merge branch 'master' into websys
thomaseizinger Sep 6, 2023
e53b665
Remove TLS
thomaseizinger Sep 6, 2023
0672fcd
Remove unnecessary log line
thomaseizinger Sep 6, 2023
7c3808c
Remove unused dev-deps
thomaseizinger Sep 6, 2023
bb7b46c
Sort lines
thomaseizinger Sep 6, 2023
c1ca67d
Update lockfile
thomaseizinger Sep 6, 2023
132bd17
Merge branch 'master' into websys
thomaseizinger Sep 13, 2023
3af8c82
Merge branch 'master' into websys
thomaseizinger Sep 17, 2023
b26efc6
Fix doc tests
thomaseizinger Sep 17, 2023
61a551e
Close connection on data overload
thomaseizinger Sep 18, 2023
9cf9e61
Fix wasm compile errors
thomaseizinger Sep 19, 2023
3ba286e
Don't pass string that says `null` but actually null
thomaseizinger Sep 20, 2023
8e59af0
Merge branch 'master' into websys
thomaseizinger Sep 20, 2023
082a364
Fix bad merge
thomaseizinger Sep 21, 2023
6fc03ae
Apply suggestions from code review
thomaseizinger Sep 27, 2023
ddbc593
Merge branch 'master' into websys
thomaseizinger Oct 9, 2023
6bd8b48
Merge branch 'websys' of github.com:libp2p/rust-libp2p into websys
thomaseizinger Oct 10, 2023
5b1e81c
Add comment
thomaseizinger Oct 10, 2023
ba8bfff
Merge branch 'master' into websys
thomaseizinger Oct 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,11 @@
- [`libp2p-pnet` CHANGELOG](transports/pnet/CHANGELOG.md)
- [`libp2p-quic` CHANGELOG](transports/quic/CHANGELOG.md)
- [`libp2p-tcp` CHANGELOG](transports/tcp/CHANGELOG.md)
- [`libp2p-tls` CHANGELOG](transports/tls/CHANGELOG.md)
- [`libp2p-uds` CHANGELOG](transports/uds/CHANGELOG.md)
- [`libp2p-wasm-ext` CHANGELOG](transports/wasm-ext/CHANGELOG.md)
- [`libp2p-websocket` CHANGELOG](transports/websocket/CHANGELOG.md)
- [`libp2p-tls` CHANGELOG](transports/tls/CHANGELOG.md)
- [`libp2p-websocket-websys` CHANGELOG](transports/websocket-websys/CHANGELOG.md)

## Multiplexers

Expand Down
22 changes: 21 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ members = [
"transports/webrtc-websys",
"transports/websocket",
"transports/webtransport-websys",
"transports/websocket-websys",
"wasm-tests/webtransport-tests",
]
resolver = "2"
Expand All @@ -71,7 +72,7 @@ rust-version = "1.65.0"

[workspace.dependencies]
futures-bounded = { version = "0.1.0", path = "misc/futures-bounded" }
libp2p = { version = "0.52.3", path = "libp2p" }
libp2p = { version = "0.52.4", path = "libp2p" }
libp2p-allow-block-list = { version = "0.2.0", path = "misc/allow-block-list" }
libp2p-autonat = { version = "0.11.0", path = "protocols/autonat" }
libp2p-connection-limits = { version = "0.2.1", path = "misc/connection-limits" }
Expand Down Expand Up @@ -111,6 +112,7 @@ libp2p-webrtc = { version = "0.6.1-alpha", path = "transports/webrtc" }
libp2p-webrtc-utils = { version = "0.1.0", path = "misc/webrtc-utils" }
libp2p-webrtc-websys = { version = "0.1.0-alpha", path = "transports/webrtc-websys" }
libp2p-websocket = { version = "0.42.1", path = "transports/websocket" }
libp2p-websocket-websys = { version = "0.2.0", path = "transports/websocket-websys" }
libp2p-webtransport-websys = { version = "0.1.0", path = "transports/webtransport-websys" }
libp2p-yamux = { version = "0.44.1", path = "muxers/yamux" }
multistream-select = { version = "0.13.0", path = "misc/multistream-select" }
Expand Down
3 changes: 2 additions & 1 deletion interop-tests/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }

[target.'cfg(target_arch = "wasm32")'.dependencies]
libp2p = { path = "../libp2p", features = [ "ping", "macros", "webtransport-websys", "wasm-bindgen", "identify"] }
libp2p = { path = "../libp2p", features = [ "ping", "macros", "webtransport-websys", "wasm-bindgen", "identify", "websocket-websys", "yamux", "noise"] }
libp2p-mplex = { path = "../muxers/mplex" }
thomaseizinger marked this conversation as resolved.
Show resolved Hide resolved
libp2p-webrtc-websys = { workspace = true }
wasm-bindgen = { version = "0.2" }
wasm-bindgen-futures = { version = "0.4" }
Expand Down
7 changes: 4 additions & 3 deletions interop-tests/chromium-ping-version.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@
"containerImageID": "chromium-rust-libp2p-head",
"transports": [
{ "name": "webtransport", "onlyDial": true },
{ "name": "webrtc-direct", "onlyDial": true }
{ "name": "webrtc-direct", "onlyDial": true },
{ "name": "ws", "onlyDial": true }
],
"secureChannels": [],
"muxers": []
"secureChannels": ["noise"],
"muxers": ["mplex", "yamux"]
}
143 changes: 110 additions & 33 deletions interop-tests/src/arch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ pub(crate) mod native {
use std::time::Duration;

use anyhow::{bail, Context, Result};
use either::Either;
use env_logger::{Env, Target};
use futures::future::BoxFuture;
use futures::FutureExt;
Expand All @@ -31,7 +30,7 @@ pub(crate) mod native {
use libp2p_webrtc as webrtc;
use redis::AsyncCommands;

use crate::{from_env, Muxer, SecProtocol, Transport};
use crate::{Muxer, SecProtocol, Transport};

use super::BoxedTransport;

Expand All @@ -47,66 +46,103 @@ pub(crate) mod native {
tokio::time::sleep(duration).boxed()
}

fn muxer_protocol_from_env() -> Result<Either<yamux::Config, mplex::MplexConfig>> {
Ok(match from_env("muxer")? {
Muxer::Yamux => Either::Left(yamux::Config::default()),
Muxer::Mplex => Either::Right(mplex::MplexConfig::new()),
})
}

pub(crate) fn build_transport(
local_key: Keypair,
ip: &str,
transport: Transport,
sec_protocol: Option<SecProtocol>,
muxer: Option<Muxer>,
) -> Result<(BoxedTransport, String)> {
let (transport, addr) = match (transport, from_env::<SecProtocol>("security")) {
(Transport::QuicV1, _) => (
let (transport, addr) = match (transport, sec_protocol, muxer) {
(Transport::QuicV1, _, _) => (
quic::tokio::Transport::new(quic::Config::new(&local_key))
.map(|(p, c), _| (p, StreamMuxerBox::new(c)))
.boxed(),
format!("/ip4/{ip}/udp/0/quic-v1"),
),
(Transport::Tcp, Ok(SecProtocol::Tls)) => (
(Transport::Tcp, Some(SecProtocol::Tls), Some(Muxer::Mplex)) => (
tcp::tokio::Transport::new(tcp::Config::new())
.upgrade(Version::V1Lazy)
.authenticate(tls::Config::new(&local_key).context("failed to initialise tls")?)
.multiplex(mplex::MplexConfig::new())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0"),
),
(Transport::Tcp, Some(SecProtocol::Tls), Some(Muxer::Yamux)) => (
tcp::tokio::Transport::new(tcp::Config::new())
.upgrade(Version::V1Lazy)
.authenticate(tls::Config::new(&local_key).context("failed to initialise tls")?)
.multiplex(muxer_protocol_from_env()?)
.multiplex(yamux::Config::default())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0"),
),
(Transport::Tcp, Ok(SecProtocol::Noise)) => (
(Transport::Tcp, Some(SecProtocol::Noise), Some(Muxer::Mplex)) => (
tcp::tokio::Transport::new(tcp::Config::new())
.upgrade(Version::V1Lazy)
.authenticate(
noise::Config::new(&local_key).context("failed to intialise noise")?,
)
.multiplex(muxer_protocol_from_env()?)
.multiplex(mplex::MplexConfig::new())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0"),
),
(Transport::Ws, Ok(SecProtocol::Tls)) => (
(Transport::Tcp, Some(SecProtocol::Noise), Some(Muxer::Yamux)) => (
tcp::tokio::Transport::new(tcp::Config::new())
.upgrade(Version::V1Lazy)
.authenticate(
noise::Config::new(&local_key).context("failed to intialise noise")?,
)
.multiplex(yamux::Config::default())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0"),
),
(Transport::Ws, Some(SecProtocol::Tls), Some(Muxer::Mplex)) => (
WsConfig::new(tcp::tokio::Transport::new(tcp::Config::new()))
.upgrade(Version::V1Lazy)
.authenticate(tls::Config::new(&local_key).context("failed to initialise tls")?)
.multiplex(muxer_protocol_from_env()?)
.multiplex(mplex::MplexConfig::new())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0/ws"),
),
(Transport::Ws, Some(SecProtocol::Tls), Some(Muxer::Yamux)) => (
WsConfig::new(tcp::tokio::Transport::new(tcp::Config::new()))
.upgrade(Version::V1Lazy)
.authenticate(
tls::Config::new(&local_key).context("failed to intialise noise")?,
)
.multiplex(yamux::Config::default())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0/ws"),
),
(Transport::Ws, Ok(SecProtocol::Noise)) => (
(Transport::Ws, Some(SecProtocol::Noise), Some(Muxer::Mplex)) => (
WsConfig::new(tcp::tokio::Transport::new(tcp::Config::new()))
.upgrade(Version::V1Lazy)
.authenticate(
noise::Config::new(&local_key).context("failed to initialise tls")?,
)
.multiplex(mplex::MplexConfig::new())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0/ws"),
),
(Transport::Ws, Some(SecProtocol::Noise), Some(Muxer::Yamux)) => (
WsConfig::new(tcp::tokio::Transport::new(tcp::Config::new()))
.upgrade(Version::V1Lazy)
.authenticate(
noise::Config::new(&local_key).context("failed to intialise noise")?,
)
.multiplex(muxer_protocol_from_env()?)
.multiplex(yamux::Config::default())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0/ws"),
),
(Transport::WebRtcDirect, _) => (
(Transport::WebRtcDirect, _, _) => (
webrtc::tokio::Transport::new(
local_key,
webrtc::tokio::Certificate::generate(&mut rand::thread_rng())?,
Expand All @@ -115,9 +151,13 @@ pub(crate) mod native {
.boxed(),
format!("/ip4/{ip}/udp/0/webrtc-direct"),
),
(Transport::Tcp, Err(_)) => bail!("Missing security protocol for TCP transport"),
(Transport::Ws, Err(_)) => bail!("Missing security protocol for Websocket transport"),
(Transport::Webtransport, _) => bail!("Webtransport can only be used with wasm"),
(Transport::Webtransport, _, _) => bail!("Webtransport can only be used with wasm"),
(Transport::Tcp | Transport::Ws, None, _) => {
bail!("Missing security protocol for {transport:?}")
}
(Transport::Tcp | Transport::Ws, _, None) => {
bail!("Missing muxer protocol for {transport:?}")
}
};
Ok((transport, addr))
}
Expand Down Expand Up @@ -154,15 +194,17 @@ pub(crate) mod native {

#[cfg(target_arch = "wasm32")]
pub(crate) mod wasm {
use anyhow::{bail, Result};
use anyhow::{bail, Context, Result};
use futures::future::{BoxFuture, FutureExt};
use libp2p::core::upgrade::Version;
use libp2p::identity::Keypair;
use libp2p::swarm::{NetworkBehaviour, SwarmBuilder};
use libp2p::PeerId;
use libp2p::{noise, yamux, PeerId, Transport as _};
use libp2p_mplex as mplex;
use libp2p_webrtc_websys as webrtc;
use std::time::Duration;

use crate::{BlpopRequest, Transport};
use crate::{BlpopRequest, Muxer, SecProtocol, Transport};

use super::BoxedTransport;

Expand All @@ -181,21 +223,56 @@ pub(crate) mod wasm {
local_key: Keypair,
ip: &str,
transport: Transport,
sec_protocol: Option<SecProtocol>,
muxer: Option<Muxer>,
) -> Result<(BoxedTransport, String)> {
match transport {
Transport::Webtransport => Ok((
Ok(match (transport, sec_protocol, muxer) {
(Transport::Webtransport, _, _) => (
libp2p::webtransport_websys::Transport::new(
libp2p::webtransport_websys::Config::new(&local_key),
)
.boxed(),
format!("/ip4/{ip}/udp/0/quic/webtransport"),
)),
Transport::WebRtcDirect => Ok((
),
(Transport::Ws, Some(SecProtocol::Noise), Some(Muxer::Mplex)) => (
libp2p::websocket_websys::Transport::default()
.upgrade(Version::V1Lazy)
.authenticate(
noise::Config::new(&local_key).context("failed to initialise noise")?,
)
.multiplex(mplex::MplexConfig::new())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0/wss"),
),
(Transport::Ws, Some(SecProtocol::Noise), Some(Muxer::Yamux)) => (
libp2p::websocket_websys::Transport::default()
.upgrade(Version::V1Lazy)
.authenticate(
noise::Config::new(&local_key).context("failed to initialise noise")?,
)
.multiplex(yamux::Config::default())
.timeout(Duration::from_secs(5))
.boxed(),
format!("/ip4/{ip}/tcp/0/wss"),
),
(Transport::Ws, None, _) => {
bail!("Missing security protocol for WS")
}
(Transport::Ws, Some(SecProtocol::Tls), _) => {
bail!("TLS not supported in WASM")
}
(Transport::Ws, _, None) => {
bail!("Missing muxer protocol for WS")
}
(Transport::WebRtcDirect, _, _) => (
webrtc::Transport::new(webrtc::Config::new(&local_key)).boxed(),
format!("/ip4/{ip}/udp/0/webrtc-direct"),
)),
_ => bail!("Only webtransport and webrtc-direct are supported with wasm"),
}
),
(Transport::QuicV1 | Transport::Tcp, _, _) => {
bail!("{transport:?} is not supported in WASM")
}
})
}

pub(crate) fn swarm_builder<TBehaviour: NetworkBehaviour>(
Expand Down
7 changes: 7 additions & 0 deletions interop-tests/src/bin/config/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ use anyhow::{Context, Result};
#[derive(Debug, Clone)]
pub(crate) struct Config {
pub(crate) transport: String,
pub(crate) sec_protocol: Option<String>,
pub(crate) muxer: Option<String>,
pub(crate) ip: String,
pub(crate) is_dialer: bool,
pub(crate) test_timeout: u64,
Expand All @@ -26,8 +28,13 @@ impl Config {
.map(|addr| format!("redis://{addr}"))
.unwrap_or_else(|_| "redis://redis:6379".into());

let sec_protocol = env::var("security").ok();
let muxer = env::var("muxer").ok();

Ok(Self {
transport,
sec_protocol,
muxer,
ip,
is_dialer,
test_timeout,
Expand Down
2 changes: 2 additions & 0 deletions interop-tests/src/bin/native_ping.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ async fn main() -> Result<()> {
config.is_dialer,
config.test_timeout,
&config.redis_addr,
config.sec_protocol,
config.muxer,
)
.await?;

Expand Down
Loading