Setting extra headers in HTTP response

[robertseliger]

First, please accept our compliments on the websockets library. It has saved us a lot of work.

In our WebSocket server we have the need to receive extra header fields in the HTTP request, We also need to include extra header fields in the HTTP response. We created a class derived from WebSocketServerProtocol so that we could override the process_request method and in so doing, deal with the extra header fields. We chose this approach because it also allowed us to decorate the websocket object with some additional attributes that are unique to our server.

We ran into an issue with being able to set extra header fields in the response. Even though we set self.extra_headers to Headers that contained our extra field, we did not see these fields in the subsequent HTTP response header. We eventually determined that the value we set self.extra_headers to is ignored in the implementation of WebSocketServerProtocol.handshake.

For now our solution is to temporarily modify server.py by adding this statement before the current line 571:

extra_headers = self.extra_headers

The problem seems to be that extra_headers is an input to handshake. It is set when handshake is called on line 130, where extra_headers is set to the then value of self.extra_headers. Subsequent changes to self.extra_headers are not captured, and therefore do not appear in the HTTP response. By adding the line of code shown above, the extra fields appear as expected.

Perhaps there is a better way to solve this problem, or perhaps there is a different way to set extra headers in the response message? In either case, we wanted to bring this issue to your attention and offer a potential solution as well.

Respectfully,

Robert Seliger
WHIM Inc.
www.whimsmart.com

[aaugustin]

If I understand your problem correctly, you may be able to solve it by passing a callable in the extra_headers argument. Then you can set appropriate response headers dynamically based on the request.

See https://github.com/aaugustin/websockets/blob/017a072705408d3df945e333e5edd93e0aa8c706/src/websockets/server.py#L575-L576

This would avoid the need to interact with internal variables (namely self.extra_headers) at a point where websockets doesn't expect it.

[robertseliger]

I spent some time trying to figure out how to leverage existing hooks in order to set the response headers. I was not successful.

The challenge is that we derived a class from WebSocketServerProtocol. We call it NxMWebSocketServerProtocol. We then implemented our own version of process_request for the derived class.

By using a derived class we are able to add our own properties to the WebSocketServerProtocol instance. And, it is within the body of process_request that we compute the values that need to be associated with the extra response headers that we use.

It is within process_request that we are accessing self.extra_headers to set the extra headers.

It seems that a potentially better approach would be to allow process_request to optionally return the response headers. Presently our return value is None, but the websockets documentation says the following:

"If process_request returns None, the WebSocket handshake continues. If it returns 3-uple containing a status code, response headers and a response body, that HTTP response is sent and the connection is closed."

What if it was possible to optionally return just the response headers and when that happens, the connection is NOT closed but processing resumes just as it does when None is returned?

[aaugustin]

If the extra_headers callable had access to the protocol instance — rather than just the request path and headers — then you'd be able to do what you want.

Extensibility features were added gradually in this area; the result doesn't make sense.

In the current state of the code, probably, you can get away with overriding the private API write_http_headers. The bad news is that this will likely break in a future version :-( The good news is that the main reason for breaking it would be to provide better extensibility :-)

[aaugustin]

Right now my plan is to provide a better API when I rewrite an asyncio layer on top of the new Sans-I/O implementation.

[aaugustin]

This is now possible in the threading-based API with the process_response hook.

[maxupp]

Apparently this hasn't made it into the asyncio server yet?
I'm trying to implement an elegant exchange of session information. A client can optionally send a session_id as a header, which I already process accordingly, but I'm struggling to provide a new session ID to the client in the handshake http response. Is there any way to do that?

[aaugustin]

It will be added by this PR: #1332

I have been working on it somewhat actively in the past couple months, so there's hope.

[aaugustin]

The new asyncio API, which provides this functionality, is now merged in the main branch (#1332).

It supports two callbacks, which you can think of as middleware:

process_request(websocket, request)
process_response(websocket, request, response)

I'm quite confident that these API support your use case. Specifically you need process_response.

See https://websockets.readthedocs.io/en/latest/reference/new-asyncio/server.html#websockets.asyncio.server.serve for details