Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add documentation for forward proxy #10443

Merged
merged 6 commits into from
Aug 5, 2021
Merged

Add documentation for forward proxy #10443

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
526eacb
Add documentation for forward proxy
dklimpel Jul 21, 2021
71c1d1a
newsfile
dklimpel Jul 21, 2021
569ac59
Apply suggestions from code review
dklimpel Jul 26, 2021
c724aee
update proxy/no proxy list
dklimpel Jul 26, 2021
e614189
Update forward_proxy.md
dklimpel Jul 26, 2021
80a5c14
Apply suggestions from code review
dklimpel Aug 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/10443.doc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add documentation for forward proxy.
dklimpel marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions docs/SUMMARY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- [Installation](setup/installation.md)
- [Using Postgres](postgres.md)
- [Configuring a Reverse Proxy](reverse_proxy.md)
- [Configuring a Forward/Outbound Proxy](setup/forward_proxy.md)
- [Configuring a Turn Server](turn-howto.md)
- [Delegation](delegate.md)

Expand Down
74 changes: 74 additions & 0 deletions docs/setup/forward_proxy.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Using a forward proxy with Synapse

You can use Synapse with a forward or outbound proxy. An example of when
this is necessary is in corporate environments behind a DMZ (demilitarized zone).
Synapse supports routing outbound HTTP(S) requests via a proxy. Only HTTP(S)
proxy is supported, not SOCKS proxy or anything else.

## Configure

The `http_proxy`, `https_proxy`, `no_proxy` environment variables are used to
specify proxy settings. The environment variable is not case sensitive.
- `http_proxy`: Proxy server to use for HTTP requests.
- `https_proxy`: Proxy server to use for HTTPS requests.
- `no_proxy`: Comma-separated list of hosts, IP addresses, or IP ranges in CIDR
format which should not use the proxy. Synapse will directly connect to these hosts.

The `http_proxy` and `https_proxy` environment variables have the form: `[scheme://][<username>:<password>@]<host>[:<port>]`
- Supported schemes are `http://` and `https://`. The default scheme is `http://`
for compatibility reasons; it is recommended to set a scheme. If scheme is set
to `https://` the connection uses TLS between Synapse and the proxy.

**NOTE**: Synapse validates the certificates. If the certificate is not
valid, then the connection is dropped.
- Default port if not given is `1080`.
- Username and password are optional and will be used to authenticate against
the proxy.

**Examples**
- HTTP_PROXY=http://USERNAME:PASSWORD@10.0.1.1:8080/
- HTTPS_PROXY=http://USERNAME:PASSWORD@proxy.example.com:8080/
- NO_PROXY=master.hostname.example.com,10.1.0.0/16,172.30.0.0/16

**NOTE**:
Synapse does not apply the IP blacklist to connections through the proxy (since
the DNS resolution is done by the proxy). It is expected that the proxy or firewall
will apply blacklisting of IP addresses.

## Connection types

The proxy will be **used** for:

- push
- url previews
- phone-home stats
- recaptcha validation
- CAS auth validation
Comment on lines +42 to +46
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this list from someplace we already have or did you look through the code?

This seems to match my memory, but curious if we should double check!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list is from there #6239

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We updated this at some point, I think

synapse/docs/sample_config.yaml

Lines 205 to 206 in 016f085

# The blacklist applies to the outbound requests for federation, identity servers,
# push servers, and for checking key validity for third-party invite events.
is the most up-to-date information.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if I understand you.
The documentation is from Dec 2020. There was a change in Jan 2021 #9084
Does the config.sample needs a info for do not block if a proxy is used?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point was that the comment from #6239 didn't take into account the changes made in #9084 so I think the current list is wrong. (It says that we don't use it for federation and identity servers, but we do now).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This really is not trivial.
I took a look in code, now:

  • proxy

    synapse/synapse/server.py

    Line 408 in bf72d10

    def get_proxied_http_client(self) -> SimpleHttpClient:
    • phone_stats_home
    • cas
    • oidc
    • ui_auth/checkers

      synapse/synapse/server.py

      Line 415 in bf72d10

      def get_proxied_blacklisted_http_client(self) -> SimpleHttpClient:
    • handlers/federation
      • "Checks whether public_key has been revoked"
    • httppusher
    • class PreviewUrlResource
  • no proxy

    synapse/synapse/server.py

    Line 428 in bf72d10

    def get_federation_http_client(self) -> MatrixFederationHttpClient:
    • keyring
    • federation/transport/client
      • "Sends federation HTTP requests to other servers"
    • identity
    • media_repository

      synapse/synapse/server.py

      Line 401 in bf72d10

      def get_simple_http_client(self) -> SimpleHttpClient:
    • generic_worker
    • module_api
    • replication
    • class IdentityHandler

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I'm being an idiot -- I was confusing the proxy with the IP blacklisting code, which of course aren't the same. 😢

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. This may happen.
But I am confused about the blacklist.
How can Synapse blacklist when the proxies does the DNS resolution? Synapse does not know the IP.
In this PR #10129 a warning should be added. But there is a function in code like:

synapse/synapse/server.py

Lines 415 to 418 in bf72d10

def get_proxied_blacklisted_http_client(self) -> SimpleHttpClient:
"""
An HTTP client that uses configured HTTP(S) proxies and blacklists IPs
based on the IP range blacklist/whitelist.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can Synapse blacklist when the proxies does the DNS resolution?

It doesn't, it only applies the blacklist if a proxy isn't being used, so get_proxied_blacklisted_http_client does the following:

  • Checks if a proxy is used, if so uses it.
  • Otherwise, doesn't use a proxy and applies the blacklist.

Pretty much anything that uses get_proxied_blacklisted_http_client or get_proxied_http_client uses the proxy.

Frankly we should probably change anywhere that manually creates SimpleHttpClient to call one of the above (that's part of fixing "make proxies work everywhere IMO).

- OpenID Connect
- federation (check public_key revocation)
dklimpel marked this conversation as resolved.
Show resolved Hide resolved

It will **not be used** for:

- Application Services
- Identity servers
- Outbound federation
- In worker configurations
- connections between workers
- connections from workers to Redis
- fetch keys from other servers
dklimpel marked this conversation as resolved.
Show resolved Hide resolved
- download remote media
dklimpel marked this conversation as resolved.
Show resolved Hide resolved

## Troubleshooting

If a proxy server is used with TLS (HTTPS) and no connections are established,
it is most likely due to the proxy's certificates. To test this, the validation
in Synapse can be deactivated.

**NOTE**: This has an impact on security and is for testing purposes only!

To deactivate the certificate validation, the following setting must be made in
[homserver.yaml](../usage/configuration/homeserver_sample_config.md).

```yaml
use_insecure_ssl_client_just_for_testing_do_not_use: true
```