Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP/3 Support #2378

Merged
merged 74 commits into from
Jun 27, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
f1c2d0e
WIP to begin http3
ahopkins Dec 12, 2021
027c8e0
Merge conflicts
ahopkins Dec 26, 2021
34fe437
Merge branch 'main' of github.com:sanic-org/sanic into http3
ahopkins Dec 26, 2021
a937977
WIP
ahopkins Dec 27, 2021
6a65222
WIP
ahopkins Dec 27, 2021
fa69294
Merge main conflicts into HTTP/3
ahopkins Jan 16, 2022
a43ae38
Merge pull request #2379 from sanic-org/http3-conflict
ahopkins Jan 16, 2022
c852406
Add aioquic and HTTP auto
ahopkins Jan 16, 2022
70a3a6f
Merge branch 'http3' of github.com:sanic-org/sanic into http3-startup
ahopkins Jan 17, 2022
65459fd
WIP
ahopkins Jan 17, 2022
cab7453
Add multi-serve for http3 and http1
ahopkins Jan 19, 2022
1bb80ba
Add spinner on startup delay
ahopkins Jan 20, 2022
96a746e
Merge branch 'main' of github.com:sanic-org/sanic into http3-startup
ahopkins Jan 20, 2022
6e3e5f1
Better exception
ahopkins Feb 17, 2022
64d4419
Add alt-svc header touchup
ahopkins Feb 17, 2022
97158d8
Add altsvs
ahopkins Feb 17, 2022
13ee4c4
Allow for TLS certs to be created on HTTP/1.1 dev servers
ahopkins Feb 21, 2022
80cc6a2
Add deprecation notice
ahopkins Feb 21, 2022
bb1bf66
Cleanup TODOs
ahopkins Feb 21, 2022
e193714
Merge pull request #2380 from sanic-org/http3-startup
ahopkins Feb 22, 2022
e447751
Cleanup merge conflicts
ahopkins Feb 22, 2022
65c02e2
Cleanup merge conflicts
ahopkins Feb 22, 2022
964e0b0
Cleanup merge conflicts
ahopkins Feb 22, 2022
067316a
Merge branch 'main' of github.com:sanic-org/sanic into http3
ahopkins Feb 22, 2022
d26d79c
Add TLS password to config
ahopkins Feb 23, 2022
06035c8
Move HTTP streaming events to receiver
ahopkins Feb 23, 2022
2262c69
Streaming send
ahopkins Feb 24, 2022
517d7d5
Setup response headers
ahopkins Feb 24, 2022
8b090f2
Merge branch 'main' of github.com:sanic-org/sanic into http3-flow
ahopkins Feb 27, 2022
b66b460
Merge conflicts
ahopkins May 11, 2022
35bbdfe
Logging
ahopkins May 11, 2022
199d6ea
Move verbosity filtering to logger
ahopkins May 11, 2022
7eb64bb
Merge branch 'main' of github.com:sanic-org/sanic into verbosity
ahopkins May 11, 2022
e98a631
Fix verbosity test on ASGI
ahopkins May 11, 2022
4d2afed
Add Sanic color
ahopkins May 11, 2022
cccf536
Merge conflicts
ahopkins May 11, 2022
425182c
WIP
ahopkins May 12, 2022
46af59b
merge conflict
ahopkins May 23, 2022
9045719
Finish flow
ahopkins May 24, 2022
92d2f41
Merge pull request #2403 from sanic-org/http3-flow
ahopkins May 24, 2022
0e85218
Merge branch 'main' of github.com:sanic-org/sanic into http3
ahopkins Jun 15, 2022
171dabb
Merge branch 'main' of github.com:sanic-org/sanic into http3
ahopkins Jun 16, 2022
dd96f1a
Get regular tests running again
ahopkins Jun 16, 2022
15d654c
Add trustme certs (#2468)
ahopkins Jun 16, 2022
c944291
Merge conflicts
ahopkins Jun 19, 2022
191d5c5
Add some unit tests
ahopkins Jun 19, 2022
872b58f
Add some unit tests
ahopkins Jun 19, 2022
1222d11
Change relative import of test client
ahopkins Jun 19, 2022
d1ddbca
TLS creators tests
ahopkins Jun 19, 2022
c2792eb
Update tests
ahopkins Jun 19, 2022
9572091
clenup
ahopkins Jun 19, 2022
1c9a037
Add TLS creator tests
ahopkins Jun 20, 2022
000a166
Add TLS creator selection test
ahopkins Jun 20, 2022
ec33f8a
Remove unneeded files
ahopkins Jun 20, 2022
f1cfc34
Cleanup config tests
ahopkins Jun 20, 2022
53bf127
Add some typeing
ahopkins Jun 20, 2022
d228ae3
Remove unnecessary coverage
ahopkins Jun 20, 2022
a8c65fc
Reduce duplicate code in server runners
ahopkins Jun 20, 2022
84fd7e0
Remove unnecessary inits
ahopkins Jun 20, 2022
b64c5b6
Remove unnecessary inits
ahopkins Jun 20, 2022
bb68e28
Fix some typing and linting issues
ahopkins Jun 20, 2022
ed840a2
Expand test coverage
ahopkins Jun 20, 2022
3d37a92
Cleanup existing tests
ahopkins Jun 20, 2022
9111f93
Increase testing coverage
ahopkins Jun 20, 2022
689c830
Add connection info and transport details
ahopkins Jun 21, 2022
ceee998
Fix 3.7 and 3.8 tests
ahopkins Jun 21, 2022
44b6b72
Fix 3.7
ahopkins Jun 21, 2022
d8c1424
Merge branch 'main' of github.com:sanic-org/sanic into http3
ahopkins Jun 26, 2022
089a8d2
Better typing and error message
ahopkins Jun 26, 2022
2f244c1
Remove unnecessary __version__
ahopkins Jun 26, 2022
ea00532
Only allow stream_id on HTTP/3
ahopkins Jun 26, 2022
93ca605
Cleanup sanic-routing imports
ahopkins Jun 26, 2022
96042fc
Setup mock test for HTTP/3 send headers
ahopkins Jun 26, 2022
653343c
Merge branch 'main' into http3
ahopkins Jun 27, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
WIP to begin http3
  • Loading branch information
ahopkins committed Dec 12, 2021
commit f1c2d0e04283f0945e923fa3a01c008213e43a0c
9 changes: 9 additions & 0 deletions sanic/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
Dict,
Iterable,
List,
Literal,
Optional,
Set,
Tuple,
Expand Down Expand Up @@ -68,6 +69,7 @@
)
from sanic.handlers import ErrorHandler
from sanic.http import Stage
from sanic.http.constants import HTTP
from sanic.log import LOGGING_CONFIG_DEFAULTS, Colors, error_logger, logger
from sanic.mixins.listeners import ListenerEvent
from sanic.models.futures import (
Expand Down Expand Up @@ -1050,6 +1052,7 @@ def run(
fast: bool = False,
verbosity: int = 0,
motd_display: Optional[Dict[str, str]] = None,
version: HTTP = HTTP.VERSION_1,
) -> None:
"""
Run the HTTP Server and listen until keyboard interrupt or term
Expand Down Expand Up @@ -1156,6 +1159,7 @@ def run(
protocol=protocol,
backlog=backlog,
register_sys_signals=register_sys_signals,
version=version,
)

try:
Expand Down Expand Up @@ -1383,6 +1387,7 @@ def _helper(
backlog: int = 100,
register_sys_signals: bool = True,
run_async: bool = False,
version: Union[HTTP, Literal[1], Literal[3]] = HTTP.VERSION_1,
):
"""Helper function used by `run` and `create_server`."""
if self.config.PROXIES_COUNT and self.config.PROXIES_COUNT < 0:
Expand All @@ -1392,6 +1397,9 @@ def _helper(
"#proxy-configuration"
)

if isinstance(version, int):
version = HTTP(version)

self.debug = debug
self.state.host = host
self.state.port = port
Expand Down Expand Up @@ -1425,6 +1433,7 @@ def _helper(
"loop": loop,
"register_sys_signals": register_sys_signals,
"backlog": backlog,
"version": version,
}

self.motd(serve_location)
Expand Down
5 changes: 5 additions & 0 deletions sanic/http/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from .constants import Stage
from .http1 import Http


__all__ = ("Http", "Stage")
25 changes: 25 additions & 0 deletions sanic/http/constants.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
from enum import Enum


class Stage(Enum):
"""
Enum for representing the stage of the request/response cycle

| ``IDLE`` Waiting for request
| ``REQUEST`` Request headers being received
| ``HANDLER`` Headers done, handler running
| ``RESPONSE`` Response headers sent, body in progress
| ``FAILED`` Unrecoverable state (error while sending response)
|
"""

IDLE = 0 # Waiting for request
REQUEST = 1 # Request headers being received
HANDLER = 3 # Headers done, handler running
RESPONSE = 4 # Response headers sent, body in progress
FAILED = 100 # Unrecoverable state (error while sending response)


class HTTP(Enum):
VERSION_1 = 1
VERSION_3 = 3
21 changes: 1 addition & 20 deletions sanic/http.py → sanic/http/http1.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
from sanic.response import BaseHTTPResponse

from asyncio import CancelledError, sleep
from enum import Enum

from sanic.compat import Header
from sanic.exceptions import (
Expand All @@ -20,29 +19,11 @@
)
from sanic.headers import format_http1_response
from sanic.helpers import has_message_body
from sanic.http.constants import Stage
from sanic.log import access_logger, error_logger, logger
from sanic.touchup import TouchUpMeta


class Stage(Enum):
"""
Enum for representing the stage of the request/response cycle

| ``IDLE`` Waiting for request
| ``REQUEST`` Request headers being received
| ``HANDLER`` Headers done, handler running
| ``RESPONSE`` Response headers sent, body in progress
| ``FAILED`` Unrecoverable state (error while sending response)
|
"""

IDLE = 0 # Waiting for request
REQUEST = 1 # Request headers being received
HANDLER = 3 # Headers done, handler running
RESPONSE = 4 # Response headers sent, body in progress
FAILED = 100 # Unrecoverable state (error while sending response)


HTTP_CONTINUE = b"HTTP/1.1 100 Continue\r\n\r\n"


Expand Down
136 changes: 136 additions & 0 deletions sanic/http/http3.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
from __future__ import annotations

from typing import TYPE_CHECKING, Callable, Dict, Optional, Union

from aioquic.h0.connection import H0_ALPN, H0Connection
from aioquic.h3.connection import H3_ALPN, H3Connection
from aioquic.h3.events import H3Event

# from aioquic.h3.events import (
# DatagramReceived,
# DataReceived,
# H3Event,
# HeadersReceived,
# WebTransportStreamDataReceived,
# )
from aioquic.quic.configuration import QuicConfiguration

# from aioquic.quic.events import (
# DatagramFrameReceived,
# ProtocolNegotiated,
# QuicEvent,
# )
from aioquic.tls import SessionTicket


if TYPE_CHECKING:
from sanic.request import Request

# from sanic.compat import Header
from sanic.log import logger
from sanic.response import BaseHTTPResponse


HttpConnection = Union[H0Connection, H3Connection]


async def handler(request: Request):
logger.info(f"Request received: {request}")
response = await request.app.handle_request(request)
logger.info(f"Build response: {response=}")


class Transport:
...


class Http3:
def __init__(
self,
connection: HttpConnection,
transmit: Callable[[], None],
) -> None:
self.request_body = None
self.connection = connection
self.transmit = transmit

def http_event_received(self, event: H3Event) -> None:
print("[http_event_received]:", event)
# if isinstance(event, HeadersReceived):
# method, path, *rem = event.headers
# headers = Header(((k.decode(), v.decode()) for k, v in rem))
# method = method[1].decode()
# path = path[1]
# scheme = headers.pop(":scheme")
# authority = headers.pop(":authority")
# print(f"{headers=}")
# print(f"{method=}")
# print(f"{path=}")
# print(f"{scheme=}")
# print(f"{authority=}")
# if authority:
# headers["host"] = authority

# request = Request(
# path, headers, "3", method, Transport(), app, b""
# )
# request.stream = Stream(
# connection=self._http, transmit=self.transmit
# )
# print(f"{request=}")

# asyncio.ensure_future(handler(request))

async def send(self, data: bytes, end_stream: bool) -> None:
print(f"[send]: {data=} {end_stream=}")
print(self.response.headers)
# self.connection.send_headers(
# stream_id=0,
# headers=[
# (b":status", str(self.response.status).encode()),
# *(
# (k.encode(), v.encode())
# for k, v in self.response.headers.items()
# ),
# ],
# )
# self.connection.send_data(
# stream_id=0,
# data=data,
# end_stream=end_stream,
# )
# self.transmit()

def respond(self, response: BaseHTTPResponse) -> BaseHTTPResponse:
print(f"[respond]: {response=}")
self.response, response.stream = response, self
return response


class SessionTicketStore:
"""
Simple in-memory store for session tickets.
"""

def __init__(self) -> None:
self.tickets: Dict[bytes, SessionTicket] = {}

def add(self, ticket: SessionTicket) -> None:
self.tickets[ticket.ticket] = ticket

def pop(self, label: bytes) -> Optional[SessionTicket]:
return self.tickets.pop(label, None)


def get_config():
config = QuicConfiguration(
alpn_protocols=H3_ALPN + H0_ALPN + ["siduck"],
is_client=False,
max_datagram_frame_size=65536,
)
config.load_cert_chain("./cert.pem", "./key.pem", password="qqqqqqqq")
return config


def get_ticket_store():
return SessionTicketStore()
76 changes: 61 additions & 15 deletions sanic/server/protocols/http_protocol.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

from typing import TYPE_CHECKING, Optional

from aioquic.h3.connection import H3_ALPN, H3Connection

from sanic.http.http3 import Http3
from sanic.touchup.meta import TouchUpMeta


Expand All @@ -11,6 +14,10 @@
from asyncio import CancelledError
from time import monotonic as current_time

from aioquic.asyncio import QuicConnectionProtocol
from aioquic.h3.events import H3Event
from aioquic.quic.events import ProtocolNegotiated, QuicEvent

from sanic.exceptions import RequestTimeout, ServiceUnavailable
from sanic.http import Http, Stage
from sanic.log import error_logger, logger
Expand All @@ -19,12 +26,35 @@
from sanic.server.protocols.base_protocol import SanicProtocol


class HttpProtocol(SanicProtocol, metaclass=TouchUpMeta):
class HttpProtocolMixin:
def _setup_connection(self, *args, **kwargs):
self._http = self.HTTP_CLASS(self, *args, **kwargs)
self._time = current_time()
try:
self.check_timeouts()
except AttributeError:
...

def _setup(self):
self.request: Optional[Request] = None
self.access_log = self.app.config.ACCESS_LOG
self.request_handler = self.app.handle_request
self.error_handler = self.app.error_handler
self.request_timeout = self.app.config.REQUEST_TIMEOUT
self.response_timeout = self.app.config.RESPONSE_TIMEOUT
self.keep_alive_timeout = self.app.config.KEEP_ALIVE_TIMEOUT
self.request_max_size = self.app.config.REQUEST_MAX_SIZE
self.request_class = self.app.request_class or Request


class HttpProtocol(HttpProtocolMixin, SanicProtocol, metaclass=TouchUpMeta):
"""
This class provides implements the HTTP 1.1 protocol on top of our
Sanic Server transport
"""

HTTP_CLASS = Http

__touchup__ = (
"send",
"connection_task",
Expand Down Expand Up @@ -70,25 +100,12 @@ def __init__(
unix=unix,
)
self.url = None
self.request: Optional[Request] = None
self.access_log = self.app.config.ACCESS_LOG
self.request_handler = self.app.handle_request
self.error_handler = self.app.error_handler
self.request_timeout = self.app.config.REQUEST_TIMEOUT
self.response_timeout = self.app.config.RESPONSE_TIMEOUT
self.keep_alive_timeout = self.app.config.KEEP_ALIVE_TIMEOUT
self.request_max_size = self.app.config.REQUEST_MAX_SIZE
self.request_class = self.app.request_class or Request
self.state = state if state else {}
self._setup()
if "requests_count" not in self.state:
self.state["requests_count"] = 0
self._exception = None

def _setup_connection(self):
self._http = Http(self)
self._time = current_time()
self.check_timeouts()

async def connection_task(self): # no cov
"""
Run a HTTP connection.
Expand Down Expand Up @@ -236,3 +253,32 @@ def data_received(self, data: bytes):
self._data_received.set()
except Exception:
error_logger.exception("protocol.data_received")


class Http3Protocol(HttpProtocolMixin, QuicConnectionProtocol):
HTTP_CLASS = Http3

def __init__(self, *args, app: Sanic, **kwargs) -> None:
self.app = app
super().__init__(*args, **kwargs)
self._setup()
self._connection = None

def quic_event_received(self, event: QuicEvent) -> None:
print("[quic_event_received]:", event)
if isinstance(event, ProtocolNegotiated):
self._setup_connection(transmit=self.transmit)
if event.alpn_protocol in H3_ALPN:
self._connection = H3Connection(
self._quic, enable_webtransport=True
)
# elif event.alpn_protocol in H0_ALPN:
# self._http = H0Connection(self._quic)
# elif isinstance(event, DatagramFrameReceived):
# if event.data == b"quack":
# self._quic.send_datagram_frame(b"quack-ack")

# pass event to the HTTP layer
if self._connection is not None:
for http_event in self._connection.handle_event(event):
self._http.http_event_received(http_event)
Loading