Automate the Release Process

[dstufft]

This issue depends on #2310, #2312, and #2313.

It would be really great to automate the entire release process. Currently there are a bunch of manual steps and the issues #2310, #2312, and #2313 capture the details of automating those steps. However that still requires that someone with the keys to release installs the dependencies for doing a release and invokes the do-the-release command. It's possible that we consider that good enough and this issue should just be closed as won't fix. However the openstack folks have what I think is a fairly nice solution, it essentially boils down to doing a release is as simple as git tag -s X.Y.Z && git push --tags. From there on our their automation does everything else.

The biggest downside to this (besides the fact we have to run a server that does the automation) is that the person doing the release only signs the git tag. They will not be signing the artifacts that get uploaded to PyPI. We can have a key that lives on the automation server that signs the artifacts before we upload them. The biggest difference between that is one could make an argument that doesn't hold the same level of guarantee as the release artifacts being signed by a person does. To be specific, this is about the gpg signatures that get uploaded to PyPI which is primarily used by people like Linux distributions to verifying the downloads from PyPI.

[pradyunsg]

Thoughts on an interactive script that does this for you?

Basically,

• Run the tests
  • If they fail, aborts.
• Asks about version bump.
• Makes a release
  • Remove pre-release from version
  • Make a git commit and tag it with this version
  • Generate artifacts + Upload
  • Version Bump (adding a pre-release ".dev" or something)
  • Make a git commit for this

[AraHaan]

You can also do like now aiohttp does and use Appveyor and Travis CI for the pypi stuff. After all it can encrypt the pypi user name and password. Same for AppVeyor so it should be no troubles doing it. @asvetlov can also help you set it up as well for automatic deployment on PyPi and it also does the tag stuff for you if you set it up right. Also reading the scripts they use with their aiohttp can help as well for the CI's.

[pradyunsg]

@AraHaan Thanks for the pointer! I'll take a look. :)

[pradyunsg]

Generate artifacts + Upload

Just to be extra clear, this step takes place on the CI servers, not locally, and is indeed triggered by the tagging of the release. At least, that's how I thought of it.

[asvetlov]

Feel free to ask me if you have any problem.
I spent some time tuning autodeployment for aiohttp and other aio libraries but now it works perfectly.

[pradyunsg]

@asvetlov Will do. ^.^

[pradyunsg]

We could adopt a process like numpy? [1]

Build the wheels on CI, upload them to some online storage, download, sign and upload.

This process could be made into a script that performs these steps, needing only input from the releaser only for signing the release - the signing key would be a parameter and gpg would ask for passphrase. That keeps the same level of guarantee for the release artifacts and the automation would handle everything except the signing...

PS: Sorry for the cross-post.

[RonnyPfannschmidt]

@dstufft i would like to propose a combination of repeatable builds and allowing other people to upload signatures, then anyone who cares can upload a signature of artifacts they created themselfes

i'd love to see pip releases signed by 5-10 people arriving at the artifacts independently and not just a single person

[dstufft]

I'm more likely to just stop uploading GPG signatures than I am to invest any effort in adding additional features to supporting it. Long term TUF is the future here and it does support N of M signatures.

[asvetlov]

Sorry, what is TUF?

[pradyunsg]

@asvetlov The Update Framework - A Framework for Securing Software Update Systems

https://theupdateframework.github.io

[pradyunsg]

What we've got post #7286 would be:

• Checkout the correct branch
• Run nox -s prepare-release -- YY.N[.P]
• File a PR, wait for CI, merge on passing CI
• Run nox -s build-release -- YY.N[.P] for generating artifacts
• Run twine upload dist/* for uploading the artifacts
• Run git push upstream YY.N

I'm keeping the upload the artifacts step separate, to allow for us to potentially add complexity there, once TUF comes through.

[xavfernandez]

File a PR, wait for CI, merge on passing CI

and push the tag :)

[pradyunsg]

Oh, I'm pushing the tag after uploading the release. (see last bullet) :P

[pradyunsg]

More things worth automating:

• get-pip.py update-and-push
• ensurepip update-and-PR

[pradyunsg]

get-pip.py update-and-push

Done in pypa/get-pip#85.

ensurepip update-and-PR

This is external to us, and contingent on CPython's triage/review process which can change. Not worth doing this.

Alright. I'm gonna say that our release process is sufficiently automated. If folks want more automation, file a new issue! :)