Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Make docs directory reflect the docs heirarchy, and fix broken links #14282

Closed
wants to merge 11 commits into from

Conversation

DMRobertson
Copy link
Contributor

@DMRobertson DMRobertson commented Oct 24, 2022

This PR moves the source markdown files so that they reflect the documentation hierarchy presented in the sidebar. (I was confused that this wasn't already the case.)

Essentially just a bunch of renames. Done in PyCharm, which fixes up links as files get moved. But it can't see all the links, so I had to manually fixup a few. To do this I used https://pypi.org/project/LinkChecker/

mdbook clean
mdbook build
cd build
linkchecker **.html
Before, on develop@386e72a22d687002f9a43b2275b1308b6a80d48c
dmr on titan in synapse/book on  develop via  v16.14.0 via 🐍 v3.10.7 (matrix-synapse-py3.10) via 🐏 12GiB/15GiB | 5GiB/8GiB 
2022-10-24 15:22:00 ✔  $ linkchecker **.html
INFO linkcheck.cmdline 2022-10-24 15:22:06,735 MainThread Checking intern URLs only; use --check-extern to check extern URLs.
LinkChecker 10.1.0
Copyright (C) 2000-2016 Bastian Kleineidam, 2010-2021 LinkChecker Authors
LinkChecker comes with ABSOLUTELY NO WARRANTY!
This is free software, and you are welcome to redistribute it under
certain conditions. Look at the file `LICENSE' within this distribution.
Read the documentation at https://linkchecker.github.io/linkchecker/
Write comments and bugs to https://github.com/linkchecker/linkchecker/issues

Start checking at 2022-10-24 15:22:06+001
10 threads active,    87 links queued,  225 links in 322 URLs checked, runtime 1 seconds
/usr/lib/python3.10/site-packages/bs4/__init__.py:435: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.
  warnings.warn(

URL        `docs/usage/configuration/config_documentation.html#saml2_config'
Name       `saml2_config.user_mapping_provider.module'
Parent URL file:///home/dmr/workspace/synapse/book/sso_mapping_providers.html, line 302, col 1
Real URL   file:///home/dmr/workspace/synapse/book/docs/usage/configuration/config_documentation.html
Check time 0.000 seconds
Result     Error: URLError: <urlopen error [Errno 2] No such file or directory: '/home/dmr/workspace/synapse/book/docs/usage/configuration/config_documentation.html'>

URL        `synapse/res/templates'
Name       `synapse/res/templates'
Parent URL file:///home/dmr/workspace/synapse/book/upgrade.html, line 1210, col 1
Real URL   file:///home/dmr/workspace/synapse/book/synapse/res/templates
Check time 0.000 seconds
Result     Error: URLError: <urlopen error [Errno 2] No such file or directory: '/home/dmr/workspace/synapse/book/synapse/res/templates'>

URL        `config_documentation.html#database'
Name       `database settings'
Parent URL file:///home/dmr/workspace/synapse/book/usage/administration/admin_api/index.html, line 163, col 15
Real URL   file:///home/dmr/workspace/synapse/book/usage/administration/admin_api/config_documentation.html
Check time 0.000 seconds
Result     Error: URLError: <urlopen error [Errno 2] No such file or directory: '/home/dmr/workspace/synapse/book/usage/administration/admin_api/config_documentation.html'>

URL        `reverse_proxy.html#synapse-administration-endpoints'
Name       `recommend'
Parent URL file:///home/dmr/workspace/synapse/book/usage/administration/admin_api/index.html, line 171, col 29
Real URL   file:///home/dmr/workspace/synapse/book/usage/administration/admin_api/reverse_proxy.html
Check time 0.000 seconds
Result     Error: URLError: <urlopen error [Errno 2] No such file or directory: '/home/dmr/workspace/synapse/book/usage/administration/admin_api/reverse_proxy.html'>

URL        `systemd-with-workers'
Name       `Systemd with Workers'
Parent URL file:///home/dmr/workspace/synapse/book/workers.html, line 290, col 1
Real URL   file:///home/dmr/workspace/synapse/book/systemd-with-workers/
Check time 0.001 seconds
Warning    [file-missing-slash] Added trailing slash to
           directory.
Result     Valid: directory

URL        `../usage/administration/admin_api'
Name       `Admin API'
Parent URL file:///home/dmr/workspace/synapse/book/admin_api/account_validity.html, line 163, col 25
Real URL   file:///home/dmr/workspace/synapse/book/usage/administration/admin_api/
Check time 0.014 seconds
Warning    [file-missing-slash] Added trailing slash to
           directory.
Result     Valid: directory

URL        `user_admin_api.html#query-user-account'
Name       `query the account'
Parent URL file:///home/dmr/workspace/synapse/book/usage/administration/admin_api/index.html, line 180, col 1
Real URL   file:///home/dmr/workspace/synapse/book/usage/administration/admin_api/user_admin_api.html
Check time 0.000 seconds
Result     Error: URLError: <urlopen error [Errno 2] No such file or directory: '/home/dmr/workspace/synapse/book/usage/administration/admin_api/user_admin_api.html'>

URL        `understanding_synapse_through_grafana_graphs.html#federation'
Name       `PDU'
Parent URL file:///home/dmr/workspace/synapse/book/usage/administration/admin_api/federation.html, line 227, col 26
Real URL   file:///home/dmr/workspace/synapse/book/usage/administration/admin_api/understanding_synapse_through_grafana_graphs.html
Check time 0.001 seconds
Result     Error: URLError: <urlopen error [Errno 2] No such file or directory: '/home/dmr/workspace/synapse/book/usage/administration/admin_api/understanding_synapse_through_grafana_graphs.html'>

URL        `../synapse/http/site.py'
Name       `site.py'
Parent URL file:///home/dmr/workspace/synapse/book/usage/administration/request_log.html, line 159, col 50
Real URL   file:///home/dmr/workspace/synapse/book/usage/synapse/http/site.py
Check time 0.000 seconds
Result     Error: URLError: <urlopen error [Errno 2] No such file or directory: '/home/dmr/workspace/synapse/book/usage/synapse/http/site.py'>

URL        `docs/reverse_proxy.html'
Name       `the reverse proxy docs'
Parent URL file:///home/dmr/workspace/synapse/book/usage/administration/admin_faq.html, line 228, col 77
Real URL   file:///home/dmr/workspace/synapse/book/usage/administration/docs/reverse_proxy.html
Check time 0.000 seconds
Result     Error: URLError: <urlopen error [Errno 2] No such file or directory: '/home/dmr/workspace/synapse/book/usage/administration/docs/reverse_proxy.html'>

Statistics:
Downloaded: 3.94MB.
Content types: 7 image, 106 text, 0 video, 0 audio, 35 application, 2 mail and 509 other.
URL lengths: min=16, max=256, avg=64.

That's it. 659 links in 659 URLs checked. 2 warnings found. 8 errors found.
Stopped checking at 2022-10-24 15:22:12+001 (5 seconds)

After: 6c2e1f4
dmr on titan in synapse/book on  dmr/docs-tidy via  v16.14.0 via 🐍 v3.10.7 (matrix-synapse-py3.10) via 🐏 12GiB/15GiB | 5GiB/8GiB 
2022-10-24 15:27:47 ✔  $ linkchecker **.html
INFO linkcheck.cmdline 2022-10-24 15:28:10,649 MainThread Checking intern URLs only; use --check-extern to check extern URLs.
LinkChecker 10.1.0
Copyright (C) 2000-2016 Bastian Kleineidam, 2010-2021 LinkChecker Authors
LinkChecker comes with ABSOLUTELY NO WARRANTY!
This is free software, and you are welcome to redistribute it under
certain conditions. Look at the file `LICENSE' within this distribution.
Read the documentation at https://linkchecker.github.io/linkchecker/
Write comments and bugs to https://github.com/linkchecker/linkchecker/issues

Start checking at 2022-10-24 15:28:10+001
/usr/lib/python3.10/site-packages/bs4/__init__.py:435: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.
  warnings.warn(
10 threads active,    83 links queued,  266 links in 359 URLs checked, runtime 1 seconds

Statistics:
Downloaded: 4.04MB.
Content types: 7 image, 106 text, 0 video, 0 audio, 22 application, 3 mail and 508 other.
URL lengths: min=16, max=256, avg=67.

That's it. 646 links in 646 URLs checked. 0 warnings found. 0 errors found.
Stopped checking at 2022-10-24 15:28:16+001 (5 seconds)

@DMRobertson DMRobertson requested a review from a team as a code owner October 24, 2022 13:16
@DMRobertson DMRobertson marked this pull request as draft October 24, 2022 13:16
David Robertson added 2 commits October 24, 2022 14:50
```
dmr on titan in synapse/book on  dmr/docs-tidy via  v16.14.0 via 🐍 v3.10.7 (matrix-synapse-py3.10) via 🐏 12GiB/15GiB | 5GiB/8GiB took 8s
2022-10-24 15:15:53 ✔  $ linkchecker **.html
INFO linkcheck.cmdline 2022-10-24 15:16:33,785 MainThread Checking intern URLs only; use --check-extern to check extern URLs.
LinkChecker 10.1.0
Copyright (C) 2000-2016 Bastian Kleineidam, 2010-2021 LinkChecker Authors
LinkChecker comes with ABSOLUTELY NO WARRANTY!
This is free software, and you are welcome to redistribute it under
certain conditions. Look at the file `LICENSE' within this distribution.
Read the documentation at https://linkchecker.github.io/linkchecker/
Write comments and bugs to https://github.com/linkchecker/linkchecker/issues

Start checking at 2022-10-24 15:16:33+001
/usr/lib/python3.10/site-packages/bs4/__init__.py:435: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.
  warnings.warn(
10 threads active,    88 links queued,  241 links in 339 URLs checked, runtime 1 seconds

Statistics:
Downloaded: 4.04MB.
Content types: 7 image, 106 text, 0 video, 0 audio, 22 application, 3 mail and 508 other.
URL lengths: min=16, max=256, avg=67.

That's it. 646 links in 646 URLs checked. 0 warnings found. 0 errors found.
Stopped checking at 2022-10-24 15:16:39+001 (5 seconds)
```
@DMRobertson DMRobertson changed the title Make docs directory reflect the docs heirarchy Make docs directory reflect the docs heirarchy, and fix broken links Oct 24, 2022
@DMRobertson
Copy link
Contributor Author

@dklimpel I'm happy to hold off on this until #14086 lands.

@clokep
Copy link
Member

clokep commented Oct 24, 2022

Fixes #11274, probably?

@DMRobertson
Copy link
Contributor Author

Fixes #11274, probably?

Yes and no. According to a comment therein, this breaks a bunch of links so might not be worth it.

@DMRobertson
Copy link
Contributor Author

Fixes #11274, probably?

Yes and no. According to a comment therein, this breaks a bunch of links so might not be worth it.

That comment: #11274 (comment)

Going to close this. @anoadragon453 has a plan here and this is premature. Andrew: feel free to reopen this if you think it's useful as-is.

@anoadragon453
Copy link
Member

Oof, yes sorry @DMRobertson. I would love to do this but I think we need some link-redirection magic to not break old links first before we do. I thought this would be a fiddly task that we would need to do manually, but it turns out mdbook actually does have built-in support for maintaining a mapping of link redirects! https://rust-lang.github.io/mdBook/format/configuration/renderers.html#outputhtmlredirect

So if we additionally had a map from old URL -> new URL for each moved entry, I would be happy to merge this.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants