Connectd throttle #7366
Open
Connectd throttle #7366
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…andoned HTLC. Started seeing these in my logs, and it's the wrong way around! Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
A fairly simple change: ccan/io will now call the underlying I/O routines repeatedly until they indicate they are unfinished, *or* fail with EAGAIN. This should make a significant difference to large nodes, which currently spend far too much time calling poll() to discover a single fd is still writable (mainly, for streaming gossip). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Changed: connectd: now should use far less CPU on large nodes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
…OUTING_SYNC. This means we do have to set the network correctly though, and also we can get query messages from lightningd which we have to filter. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We currently stream gossip as fast as we can, even if they start at timestamp 0. Instead, use a simple token bucket filter and only let them have 1MB per second (500 bytes per second for testing). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Protocol: connectd: we now throttle outgoing gossip at 1MB/second per peer.
gossipd no longer connects directly to per-peer daemons, so remove old file. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is more efficient in a few ways: 1. It's trivial to get to the end of the gossip_store, we don't have to iterate. 2. It tends to be mmaped so we don't have to call pread(). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was removed from the spec on Apr 25, 2022. We stopped ever sending them in 0.12.0 (2022-08-23). Now we remove receive support. Changelog-Protocol: Removed support for zlib-compressed short-channel-ids in query responses. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This basically means moving the code from gossipd to connectd to handle these queries. This will get connectd have finer control over ratelimiting them. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
A bit tricky, since we get more than one message at a time. However, this just means we go over quota for a bit, and will get caught when those are sent (we do this for a single message already, so it's not that much worse). Note: this not only limits sending, but it limits the actuall query processing, which is nice. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
cdecker
approved these changes
Jun 23, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Builds on #7365 and #7364
This makes connectd serve gossip and gossip query responses with a ratelimit of 1MB/second. This at least puts some limit on how fast they stream gossip from us. It also includes a few optimizations and removes support for decoding compressed scids (dropped from spec and implementations in 2022).