[Epic] VPN support

[gbraad]

No description provided.

[jlynchMicron]

@gbraad is this a coming fix for crc in the immediate future? I have cisco anyconnect installed by my company that I cannot disable, and I think it is causing me issues during "crc start" similar to this issue #908

[gbraad]

No timeframe is given as issues are encountered while implementing

[gbraad]

Planned to be reviewed again.

[gbraad]

@guillaumerose Do you have any suggestions on how to tackle this?

[guillaumerose]

Docker Desktop solves this by using vpnkit.

vpnkit runs as user on the host and connects to a small daemon inside the VM using named pipes/vsock. The daemon in the VM is in fact a tap interface. The daemon transfers all the traffic to vpnkit so that vpnkit handles all the traffic as the user.

In this architecture, vpnkit is a client because of issue with vsock on older version of Windows. It could be reversed now.

[guillaumerose]

I started an experiment in https://github.com/guillaumerose/gvisor-tap-vsock. It reproduces vpnkit without ocaml code. For the moment, it's more a toolkit to solve the VPN issue than the solution itself.

[gbraad]

We should break down work in new tasks as the spike has concluded.

Some time ago we noted what was needed:

• Make the client more reliable: retry loop if connection fails + automatic setup of the network (ip addr etc.) [side note: how to give network configuration/dhcp? how to disable it when not needed? ignition ?]
• Package the client part as a container, add it in snc. [side note: how to patch a bundle without having to relaunch the full snc process?]
• Add the needed registry key for Windows in crc setup
• Create a a new crc subcommand that will run the host part OR make crc start --with-experimental-networking a never ending running task [side note: 2nd option is probably easier for user]
• Add services if required (dns, transparent proxy)

I know it is boring, but let's break up tasks and assign to platforms. This will aid in testing, as we know what is implemented and perhaps even handover some of the work.

[guillaumerose]

crc-org/snc#234 addresses the first two points.
For the 4th point, the host part is part of the daemon and the config to enable it is network-mode=vsock #1526

[gbraad]

Related pull requests

• Add gvisor-tap-vsock daemon snc#234
• Add userland tcpip stack for the VM #1526
• Make VMNet optional machine#43
• Make VMNet optional machine-driver-hyperkit#20
• Add optional Virtual Switch machine#46

[guillaumerose]

First bits are in master branch. It can be used on the 3 platforms.

How to test it:

• cleanup everything, ~/.crc and hosts file
• crc config set network-mode vsock
• run crc setup
• run crc daemon in a second terminal
• run crc start as usual.

What is still missing:

• DNS records for apps created by user is not working. DNS records and port forwarding are hardcoded for vsock network mode #1685
  Workaround: manually add the dns record in the hosts file.
• Port forwarding is static. Only VM port 443, 22 (accessible through port 2222 on the host) and 6443 are reachable.
• Communication still happen between crc and his components using localhost and random ports. Only use unix domain socket and named pipes for internal communication #1686

What can be improved:

• Port forwarding binds all IP on the host. We could restrict that to localhost.
• hosts file is not cleaned up. See [BUG] DNS records are not cleaned up in /etc/hosts after crc cleanup #1328 / Wrap goodhosts under our own administrator CLI tool #1652
• macOS dialog prompt when running the daemon. See Running crc daemon on macOS prompts Do you want the application to accept incoming network connections? #1667

[cfergeau]

cleanup everything, ~/.crc and hosts file

which means crc delete; crc cleanup; rm -rf ~/.crc, and finally remove all xxx.crc.testing entries from /etc/hosts

[fasho]

bump

[gbraad]

@fasho Please look at https://github.com/code-ready/crc/wiki/VPN-support--with-an--userland-network-stack

[guillaumerose]

VPN support is available by default on macOS and Linux.
It will be activated by default by the MSI on Windows in the next release. Until then, please refer to https://github.com/code-ready/crc/wiki/VPN-support--with-an--userland-network-stack

Still, we are missing end-to-end tests for this.