Accepting back-pressure from slow websocket clients

[njsmith]

Long story short

If a websocket client is slow to read data from the network -- perhaps because of a network failure, or because of a malicious client -- then AFAICT there's no way for an aiohttp websocket server to detect this and slow down sending or drop that client. Instead, the server will just keep sending data and the data will queue up indefinitely inside the server. In some cases, such as a websocket server that sends out an ongoing stream of updates without reading responses (e.g.: a chat server or something like a live twitter feed), then this makes it possible to trivially trigger memory leaks on servers by sending them just a few packets.

Steps to reproduce

Point this client:

import sys, asyncio, aiohttp

URL = sys.argv[1]

async def constipate(url):
    session = aiohttp.ClientSession()
    async with session.ws_connect(url) as ws:
        await asyncio.sleep(100000000)

asyncio.get_event_loop().run_until_complete(constipate(URL))

At this server:

from aiohttp import web

async def endless_update_stream(request):
    ws = web.WebSocketResponse()
    await ws.prepare(request)
    while True:
        ws.send_str("x" * 100000)

app = web.Application()
app.router.add_get("/", endless_update_stream)
web.run_app(app, port=8081)

But remember to hit control-C before the server runs your machine out of memory and it starts swapping to death.

Solution

The simplest solution would be to add an async drain method to websocket handlers, similar to asyncio.StreamWriter. Probably a better solution would be to make send_str and friends async methods so that users didn't have to remember to call drain explicitly, but of course this has API compatibility issues.

[thehesiod]

I may be off but I think this blog post may be worth reading: https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/

[fafhrd91]

should be fixed in master

[fafhrd91]

Thanks for pointing to the problem

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.