Use binary message RPC protocol for plugins #11093
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Thomas Mäder <tmader@redhat.com>
Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. Currently we have to encode/decode between binary data to handle RPC calls from a plugin context. Aligning the two protocols and zero-copy tunneling of RPC messages is planned for a follow-up PR. Contributed on behalf of STMicroelectronics. Closes eclipse-theia#10684
Refactor plugin-ext rpc protocol to reuse the new binary message-rpc protocol. - - Remove custom RPC message encoding and handling reuse message-rpc -- Implement `QueuingChannelMultiplexer`that queues messages and sends them accumlated on the next process.tick (replaces the old `Multiplexer` implementation -- Refactors proxy handlers and remote target handlers -- Use `Channel` instead of `MessageConnection` for creating new instances of `RPCProtocol` -- Implement special message encoders and decoders for the plugin communication - Adapt the `HostedPluginServer` and `HostedPluginClient` API to send/receive messages in binary format instead of strings. - Implement a message protocol for process pipe streams to be able to send messages in binary format between the hosted-plugin-process and the plugin-host. Co-authored-by: Lucas Koehler <lkoehler@eclipsesource.com> Part of eclipse-theia#10684 Contributed on behalf of STMicroelectronics
1 task
|
why not compress json to binary data and send the data on socket? Or just wrap the binary data reading & writing & parsing inside WebsocketChannel ? It should be easier and no api broken. |
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
May 13, 2022
Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Contributed on behalf of STMicroelectronics. Closes eclipse-theia#10684
Hi, thanks for your input. This PR is based on #11011 and the code your are referring to is part of this parent PR. The question whether it would be better do not directly expose binary channels and make the binary encoding/decoding an implementation details of the corresponding channel implementation (Websocket, IPC, Electron IPC etc.) has also been raised there: #11011 (comment) If you have any more input/questions please comment in the main PR. |
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
May 17, 2022
Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Contributed on behalf of STMicroelectronics. Closes eclipse-theia#10684
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
May 17, 2022
Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Contributed on behalf of STMicroelectronics. Closes eclipse-theia#10684 Co-authored-by: Thomas Mäder <tmader@redhat.com>
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
May 17, 2022
Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Contributed on behalf of STMicroelectronics. Closes eclipse-theia#10684 Co-authored-by: Thomas Mäder <tmader@redhat.com>
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
May 25, 2022
Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Contributed on behalf of STMicroelectronics. Closes eclipse-theia#10684 Co-authored-by: Thomas Mäder <tmader@redhat.com>
JonasHelming
pushed a commit
that referenced
this pull request
May 25, 2022
…ns) (#11011) Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also #10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See #11093) Contributed on behalf of STMicroelectronics. Closes #10684 Co-authored-by: Thomas Mäder <tmader@redhat.com>
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
May 31, 2022
…ns) (eclipse-theia#11011) Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Contributed on behalf of STMicroelectronics. Closes eclipse-theia#10684 Co-authored-by: Thomas Mäder <tmader@redhat.com>
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
Jun 1, 2022
…ns) (eclipse-theia#11011) Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Fix critical bugs - eclipse-theiaGH-11196 Remove dev/debug logoutput from IPCChannel - eclipse-theiaGH-11199 Refactor connection provider and channel multiplexer to properly handle a reconnecting backend - Remove console log in websocket-channel if the socket is not connected. Otherwise we end up in an endless loop. - Ensure that channels & RpcProtocol instances proplery dispose all resources if the backend disconnects. - Ensure that all previously open channels and RpcProtocol instances are properly restored once the backend reconnects. - eclipse-theia#11203 Ensure that debugging is handled gracefully (implicitly fixed with the fix for eclipse-theia#11199) - Remove dependency to `reconnecting-websocket` which is no longer needed since the swap to socket.io Fixes eclipse-theia#11196 Fixes eclipse-theia#11199 Contributed on behalf of STMicroelectronics. Co-authored-by: Thomas Mäder <tmader@redhat.com>
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
Jun 1, 2022
…ns) (eclipse-theia#11011) Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Fix critical bugs - eclipse-theiaGH-11196 Remove dev/debug logoutput from IPCChannel - eclipse-theiaGH-11199 Refactor connection provider and channel multiplexer to properly handle a reconnecting backend - Remove console log in websocket-channel if the socket is not connected. Otherwise we end up in an endless loop. - Ensure that channels & RpcProtocol instances proplery dispose all resources if the backend disconnects. - Ensure that all previously open channels and RpcProtocol instances are properly restored once the backend reconnects. - eclipse-theia#11203 Ensure that debugging is handled gracefully (implicitly fixed with the fix for eclipse-theia#11199) - Remove dependency to `reconnecting-websocket` which is no longer needed since the swap to socket.io Fixes eclipse-theia#11196 Fixes eclipse-theia#11199 Contributed on behalf of STMicroelectronics. Co-authored-by: Thomas Mäder <tmader@redhat.com>
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
Jun 2, 2022
…ns) (eclipse-theia#11011) Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Fix critical bugs - eclipse-theiaGH-11196 Remove dev/debug logoutput from IPCChannel - eclipse-theiaGH-11199 Refactor connection provider and channel multiplexer to properly handle a reconnecting backend - Remove console log in websocket-channel if the socket is not connected. Otherwise we end up in an endless loop. - Ensure that channels & RpcProtocol instances proplery dispose all resources if the backend disconnects. - Ensure that all previously open channels and RpcProtocol instances are properly restored once the backend reconnects. - eclipse-theia#11203 Ensure that debugging is handled gracefully (implicitly fixed with the fix for eclipse-theia#11199) - Remove dependency to `reconnecting-websocket` which is no longer needed since the swap to socket.io Fixes eclipse-theia#11196 Fixes eclipse-theia#11199 Contributed on behalf of STMicroelectronics. Co-authored-by: Thomas Mäder <tmader@redhat.com>
tortmayr
added a commit
to eclipsesource/theia
that referenced
this pull request
Jun 2, 2022
…ns) (eclipse-theia#11011) Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See eclipse-theia#11093) Fix critical bugs - eclipse-theiaGH-11196 Remove dev/debug logoutput from IPCChannel - eclipse-theiaGH-11199 Refactor connection provider and channel multiplexer to properly handle a reconnecting backend - Remove console log in websocket-channel if the socket is not connected. Otherwise we end up in an endless loop. - Ensure that channels & RpcProtocol instances proplery dispose all resources if the backend disconnects. - Ensure that all previously open channels and RpcProtocol instances are properly restored once the backend reconnects. - eclipse-theia#11203 Ensure that debugging is handled gracefully (implicitly fixed with the fix for eclipse-theia#11199) - Remove dependency to `reconnecting-websocket` which is no longer needed since the swap to socket.io Fixes eclipse-theia#11196 Fixes eclipse-theia#11199 Contributed on behalf of STMicroelectronics. Co-authored-by: Thomas Mäder <tmader@redhat.com>
JonasHelming
pushed a commit
that referenced
this pull request
Jun 2, 2022
…ns) (#11011) (#11228) Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also #10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Remove 'vscode-ws-jsonrpc' depdency from "@theia/core/shared". - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. This will be addressed in a follow-up PR. (See #11093) Fix critical bugs - GH-11196 Remove dev/debug logoutput from IPCChannel - GH-11199 Refactor connection provider and channel multiplexer to properly handle a reconnecting backend - Remove console log in websocket-channel if the socket is not connected. Otherwise we end up in an endless loop. - Ensure that channels & RpcProtocol instances proplery dispose all resources if the backend disconnects. - Ensure that all previously open channels and RpcProtocol instances are properly restored once the backend reconnects. - #11203 Ensure that debugging is handled gracefully (implicitly fixed with the fix for #11199) - Remove dependency to `reconnecting-websocket` which is no longer needed since the swap to socket.io Fixes #11196 Fixes #11199 Contributed on behalf of STMicroelectronics. Co-authored-by: Thomas Mäder <tmader@redhat.com>
|
Closed in favor of #11261 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What it does
Note: Includes the new protocol from #11011 and, thus, should be merged afterwards.
QueuingChannelMultiplexerthat queues messages and sends them accumulated on the nextprocess.tick(replaces the old Multiplexer implementation)Channelinstead ofMessageConnectionfor creating new instances ofRPCProtocolHostedPluginServerandHostedPluginClientAPI to send/receive messages in binary format instead of strings.hosted-plugin-processand theplugin-host.Contributed on behalf of STMicroelectronics
Part of #10684
How to test
Review checklist
Reminder for reviewers