Add changelog to the Sphinx docs

[webknjaz]

This patch enables Sphinx to include the changelog document as a top-level web page. Preview:

Live demo: https://pip-tools--1469.org.readthedocs.build/en/1469/changelog/

P.S. The only goal of this PR is to include CHANGELOG.md into the Sphinx setup.

Contributor checklist

• Provided the tests for the changes.
• Assure PR title is short, clear, and good to be included in the user-oriented changelog
• Assure one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
• Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

[jdufresne]

IMO, we'd be better off converting the top-level document to rst and then moving it into the docs directory. It would then become a first class document in the docs infrastructure.

[ssbarnea]

@jdufresne I find markdown more contributor friendly than ReST, especially for less experienced python developers. As sphinx already got decent md support I see a migrations towards md everywhere as better in long term.

I personally can write both, but I based on what I seend as incoming pull requests from others, people struggle less writing markdown than rst.

[jdufresne]

As sphinx already got decent md support I see a migrations towards md everywhere as better in long term.

I still prefer rst but am not against switching to md.

Either way, this sounds like an orthogonal issue to me. First step would be moving the changelog into the official docs in whatever format they are currently being written in.

[webknjaz]

I also prefer RST but it's out of the scope here. Plus it's probably hard with the current integration that relies on GH PRs, hence MD.

[jdufresne]

@webknjaz I'm not clear on your position, can you clarify?

Are you saying moving the changelog to the docs/ directory as rst is out of scope? If so, I disagree. If we're going to better integrate the changelog then I think we should do so completely by moving it fully to the docs.

Plus it's probably hard with the current integration that relies on GH PRs, hence MD.

Can you be explicit what this means? I don't know what you're referring to.

We can always rework or reevaluate tools if they are making other work more difficult.

[webknjaz]

It's out of the scope of this PR. Because it's about including one file into the sphinx setup and nothing else.

Fully remaking the changelog generation procedure is something that does not fit within the scope.

[jdufresne]

It's out of the scope of this PR. Because it's about including one file into the sphinx setup and nothing else.

I still disagree with this statement. I think it is a great move to bring the changelog into the docs, but I think it should be moved as an rst file so it matches the existing docs conventions. That to me feels like a more complete change. That one file should match the file type of the of the other docs, IMO.

Fully remaking the changelog generation procedure ...

Can you provide more detail about this?

[webknjaz]

It's out of the scope of this PR. Because it's about including one file into the sphinx setup and nothing else.

I still disagree with this statement. I think it is a great move to bring the changelog into the docs, but I think it should be moved as an rst file so it matches the existing docs conventions. That to me feels like a more complete change. That one file should match the file type of the of the other docs, IMO.

I don't understand how you can disagree with what I have in mind for this change. It's not like this was a clearly stated task to convert between formats. It's a continuation of the effort of connecting more files with Sphinx.
There was no Sphinx setup at all a few months ago. Zero. So there's no docs conventions that exist at this point. The only few documents that existed before we added a Sphinx skeleton were written in Markdown. So if we were to talk about conventions, I'd argue that the convention is to use Markdown. I didn't want us to be limited with one format or another, especially with zero docs, so the skeleton got a MyST integration from the very beginning. And the de-facto setup supports both RST and Markdown (MyST allows using RST directives within MD, FYI). That is the convention.
Still, there's no real docs content except for the README and CONTRIBUTING inclusion in the Sphinx docs. That's it. It was made so that the skeleton PR remains small.
And now this PR's only goal is to include another document in Sphinx. Not change its format, not change its contents. Nothing at all, except make sure that it's integrated and linted with Sphinx.
I have no intention of expanding the scope because I don't have time for anything else at this point. Besides, this PR leaves the repository in a better state. Converting the changelog procedures is something you could work on separately, once this is merged.

P.S. If anything, the only RST file in the repo may need to be converted to MD too if we'll want to go for partial (or fragmented) inclusion of README instead of including the whole document.

Fully remaking the changelog generation procedure ...

Can you provide more detail about this?

The current changelog generator relies on the PR titles/descriptions that show up on the web interface of GitHub. GitHub only renders Markdown in PRs so whoever fills out their titles/descriptions provides changelog fragments in Markdown. There's no way to source RST from these places. It all relies on a GitHub App/Action integration that doesn't do anything other than putting that content into a file. If we were to transition to RST, somebody would need to automate some post-conversion or migrate to a completely different changelog management workflow (like using Towncrier).

[atugushev]

Is there a chance to hide the list of versions in the left sidebar? That info is not so important to be in the foreground.

[jdufresne]

It's a continuation of the effort of connecting more files with Sphinx.

This is an effort that I very much agree with! Thanks for contributing it. However, I think we should continue using the Sphinx default format and should match the existing documents there. Mixing md and rst is something I think is unnecessary and easy to avoid.

I have no intention of expanding the scope because I don't have time for anything else at this point. Besides, this PR leaves the repository in a better state. Converting the changelog procedures is something you could work on separately, once this is merged.

I don't necessarily agree this is a better state. Dealing with multiple formats unnecessarily is a complexity level (however minor you may consider) that doesn't need to exist. I think we should wait to land this until that work can be completed.

P.S. If anything, the only RST file in the repo may need to be converted to MD too if we'll want to go for partial (or fragmented) inclusion of README instead of including the whole document.

I am not advocating for this. Instead, I think we should move towards Sphinx and community defaults which is typically rst. It is a fine format and well understood in the Python docs community.

---

Additionaly, I think we should drop the changelong builder for curated docs. In my experience, the changelog builder is unnecessary and more trouble than it is worth. Changelogs are easy enough to write manually.

[jdufresne]

I converted the single Sphinx md doc to rst in #1474 using pandoc. We can do the same here.

[webknjaz]

Is there a chance to hide the list of versions in the left sidebar? That info is not so important to be in the foreground.

This should be possible by increasing the level of the version titles.

[ssbarnea]

Is there a chance to hide the list of versions in the left sidebar? That info is not so important to be in the foreground.

This should be possible by increasing the level of the version titles.

It looks like "Changelog" title is using same level as the versions. Going with it up may save you from altering level for each version.

[webknjaz]

It looks like "Changelog" title is using same level as the versions. Going with it up may save you from altering level for each version.

Nope. Version titles already use <h1> and there's no <h0> so I'll have to alter every title.

[ssbarnea]

The indentation is better fixed in follow-up inside changelog.md file by adding one extra # to each header.

[webknjaz]

@atugushev fixed: https://pip-tools--1469.org.readthedocs.build/en/1469/changelog/

[atugushev]

Neat! 👍🏻

[webknjaz]

It's a continuation of the effort of connecting more files with Sphinx.

This is an effort that I very much agree with! Thanks for contributing it. However, I think we should continue using the Sphinx default format and should match the existing documents there. Mixing md and rst is something I think is unnecessary and easy to avoid.

I'm active in various Sphinx and tech writer communities (Write The Docs Slack and Sphinx's tracker, in particular) and created, contributed, and maintained many docs sites. While technically the default still offers RST, there's a lot of talk about the downsides of docutils pushing people to explore other things. There's also an effort of producing tutorials lowering the entry barrier for newcomers. Most people agree that MD is easier and there were talks about making MyST a first-class citizen in the Sphinx docs. That got postponed, pending discussions, but is actively supported by many folks from different communities.

I do like RST but it's objectively more complicated. It was suggested in some PyCon talks that it's best to keep the door open for the contributors who don't know RST (and all its complexities). Back then, however, MD support in Sphinx was poor and now, with MyST, it isn't. RST is coming from CPython and is still mostly only known by folks in the Python community. When new people come to contribute, they don't know RST. Even with Python projects, there's subcommunities (outside of my or your bubbles) that prefer MD. One of them, where I see MD most, is the scientists. Others are people coming from other languages and are just used to writing docs in MD.

So no, I cannot call RST a go-to choice anymore. I personally use it where I can but I'm not going to exclude people based on this. Besides, there are currently no complex docs in this project that could clearly demonstrate the superiority of RST over MD.

I have no intention of expanding the scope because I don't have time for anything else at this point. Besides, this PR leaves the repository in a better state. Converting the changelog procedures is something you could work on separately, once this is merged.

I don't necessarily agree this is a better state. Dealing with multiple formats unnecessarily is a complexity level (however minor you may consider) that doesn't need to exist. I think we should wait to land this until that work can be completed.

By better state, I mean having the page shown in Sphinx vs not having it shown in Sphinx. It's orthogonal to what formats are being used.

P.S. If anything, the only RST file in the repo may need to be converted to MD too if we'll want to go for partial (or fragmented) inclusion of README instead of including the whole document.

I am not advocating for this. Instead, I think we should move towards Sphinx and community defaults which is typically rst. It is a fine format and well understood in the Python docs community.

See above for my comments on why it is currently incorrect to call RST the only Sphinx community default.

Additionaly, I think we should drop the changelong builder for curated docs. In my experience, the changelog builder is unnecessary and more trouble than it is worth. Changelogs are easy enough to write manually.

This is up to the person who would be stuck with compiling the changelog file manually. I haven't seen an RM enjoying this.

FWIW with two approvals, I hope @atugushev could merge this now.

[jezdez]

Hey all, in case it matters, I'm in favor of supporting Markdown as an alternative to rST (via MyST) since it lowers the barrier for writing documentation and that's a core goal of Jazzband.

I know rST still has inherent advantages, especially for cross-linking and extensibility, but we also can't ignore the rise of Markdown as the defacto markup of the GitHub & notebook era.

Just my 2 cents obviously, and the devil is in the detail, as always.

PS: Please don't migrate the docs from rST to Markdown as a whole based on this comment, that's a whole other story.

[ssbarnea]

@jezdez As you said, we can still live with a combination of markdown and rst. We will see in the future if conversion would be desired or not but I do not find it as a priority.

BTW, I really love how the changelog looks inside the docs. A clear benefit!

[webknjaz]

@jezdez thanks, that's what I was thinking:

1. Inclusivity is important.
2. We shouldn't force-convert existing content without a compelling reason.
3. MyST is something beyond traditional Markdown because it allows making use of RST directives via its extended syntax. So it's not even a limiting factor.
4. People who like RST should keep using it, this effort is in no way attempting to limit this.

Since it's +3 vs -1 I'd like to go ahead and do not block this MR on this requested change. I hope you don't mind @jdufresne. Feel free to open an issue to discuss moving out to RST format.

[webknjaz]

(rebased after merging #1480)