Add an UDP listening endpoint to prove reachability

[tomaka]

With the intent to solve #564

The idea is to add a small UDP listening endpoint that answers requests. Then we can tackle #564 by sending out these requests.

cc'ing @romanb @arkpar @kirushik

[tomaka]

I can see two possible ways to design the protocol:

• We can copy the ICMP PONG message, or the Kademlia PING message, and just make the sender emit an arbitrary payload which the recipient answer echoes back.

• Or we can provide more security by proving that the node owns its key.

If we go the second option, one quick brainstorming I went with a packet containing:

• A flag indicating whether this is a request or a response
• A timestamp of the emission
• Some implementation-specific bytes
• The node's network public key.
• A signature of hash(concat(flag, timestamp, random_bytes)) using that key.

Recipient sends back an answer containing the bytes that the sender sent.

[romanb]

Something that I don't understand is the following: The addresses reported by a peer are multiaddresses. A multiaddress specifies the transport protocol to use. To check whether the address is reachable, we surely must use the transport specified in the address? I.e. in the context of #564 I don't even see how there can be such a choice of transport protocol?

[tomaka]

I.e. in the context of #564 I don't even see how there can be such a choice of transport protocol?

To me, that's why we're going to put this code in Substrate and not libp2p.
Substrate enforces that the transport is TCP/IP (or WebSockets).
Considering that Substrate doesn't support any other protocol, if we receive a multiaddress that isn't /ip/.../tcp/...(/ws), we can instantly consider it unreachable without going through any UDP pinging.

On the long term, though, we can probably remove this feature once we switched to QUIC?

[kirushik]

(Here I assume that dialing UDP on the same port as your TCP listeners is a good design and a feature we want to keep; it's far from obvious, but for now let's ignore that)

There are two major threat categories here:
a) malicious peer M would falsely identify to our node N, forcing N to send undesired UDP pings to a victim node V;
b) malicious peer M would ping N with a return address of their choice, so N would send undesired UDP pongs to V.

a can't be fully mitigated, since sending UDP packets to sometimes-unreachable endpoints is the explicit goal of the design; we should still take care of the following:

• we should set a reasonable limit of the total amount of addresses the peer can send us through Identify;
• we shouldn't retry pings;
• we shouldn't ping the same address too frequently even if it was advertised from multiple (potentially malicious) peers;
• we should rate-limit pinging, ideally by both "maximal pings sent per second" and "maximal allowed pings in-flight" metrics (we must clean up the in-flight pings tracker after a while, though — otherwise M can "ping-starve" us, occupying all the inflight tracking slots with unreachable addresses);
• we shouldn't ping addresses in private subnetworks (doesn't make much sense if we're talking about globally-visible DHT records anyway);

The same list also improves our defenses against b when applied to PONGs (since those are general measures protecting an ability to send UDP from abuse), but we would need some additional points:

• PONG packet should be shorter than PING, otherwise UDP amplification attacks would be possible;
• we need to have some protection against PING packets replay — probably by adding an expiration field, and ignoring pings with both expiration time in the past and too far (say, >10-15 seconds) in the future — careful, this would make the identification+pring protocol sensitive to accurate clock sync.

[burdges]

If I understand, this proves nothing about the identity of the node reached, except perhaps that substrate runs there, right?

[arkpar]

@burdges pongs should be signed by the key that matches expected peer id.

devp2p design has a few strong points on rate limiting and replay protection already.
For reference:
Current version that includes timestamp in the ping message:
https://github.com/ethereum/devp2p/blob/master/discv4.md
Proposed version:
https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md
I'd recommend whoever's going to work on this to study devp2p discovery first.

[twittner]

Given that we support multiple listen addresses we would need to bind one UDP socket to each address in order to answer PINGs to those addresses if they are reported via identify.
A PONG over UDP would also not prove that the service is reachable over TCP, so at best we get some indication that the address might work. In other words we just want better heuristics to decide which addresses to consider and which ones to ignore and to achieve that we could filter addresses based on the address ranges they belong to. This could be done in libp2p-identify which would produce more relevant results.

[tomaka]

To expand on the previous comment, the problem we have is: if a node connects to us through TCP, how do you know which UDP port to ping?

Do we hard-code a specific port into the client? This means that you could for example no longer start two nodes on the same machine, and it is generally frown upon to not make ports configurable.

But if we make the port configurable, then we need to somehow communicate to remotes which UDP port to try. This means designing a new protocol just for that.

[burdges]

There are claims that merely opening a QUIC connection might prove heavier than you'd like, but computationally the crypto for opening a QUIC connection should only cost twice the unencrypted signed message proposed here. We could gain other stuff from doing that if either idling QUIC connections proves cheap or if we used them to initiate 0-RTT.