Improve release checklist

packages/obs-release.sh

@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+
+# $ ./obs-release.sh PROJECT PACKAGE VERSION
+#
+# This script uses osc to create (or check out) a branch of PROJECT in your OBS
+# home project, update the release version and commit the changes to OBS.
+#
+# If `osc` command is not available, the script automatically runs the docker
+# image crystallang/osc and executes itself in the container.
+#
+# Parameters:
+# * `PACKAGE`: OBS base package (e.g. `devel:languages:crystal`)
+# * `PROJECT`: OBS project in `PACKAGE` (e.g. `crystal`)
+# * `VERSION`: Release version (e.g `1.1.1`)
+#
+# Requirements:
+# * packages: osc build which
+# * environment variables:
+#   `OBS_USER`: OBS username
+#   `OBS_PASSWORD`: OBS password (only necessary if ~/.oscrc is missing)
+
+set -eu
+
+if ! command -v osc > /dev/null; then
+  exec docker run --rm -it \
+    -e OBS_USER=${OBS_USER:-} \
+    -e OBS_PASSWORD=${OBS_PASSWORD:-} \
+    -v $(pwd):/workspace -w /workspace \
+    crystallang/osc /workspace/$0 $@
+fi
+
+PROJECT=$1
+PACKAGE=$2
+VERSION=$3
+
+if [ ! -f ~/.oscrc ]; then
+  ./obs-setup.sh
+fi
+
+# Checkout OBS package
+LOCAL_BRANCH_FOLDER="home:$OBS_USER:branches:$PROJECT/$PACKAGE"
+
+if [ -d "${LOCAL_BRANCH_FOLDER}" ]; then
+  pushd "${LOCAL_BRANCH_FOLDER}"
+  osc up
+else
+  osc branchco "$PROJECT" "$PACKAGE"
+  pushd "${LOCAL_BRANCH_FOLDER}"
+fi
+
+# Update version in *.dsc and *.spec
+sed -i -e "s/^Version: .*/Version: ${VERSION}-1/" *.dsc
+sed -i -e "s/^DEBTRANSFORM-TAR: .*/DEBTRANSFORM-TAR: ${VERSION}.tar.gz/" *.dsc
+sed -i -e "s/^Version: .*/Version: ${VERSION}/" *.spec
+sed -i -e "s/VERSION := .*/VERSION := ${VERSION}/" debian.rules
+
+# Commit changes to OBS
+message="Release $VERSION"
+osc vc -m "$message"
+osc diff
+
+# Commit
+osc commit -m "$message" --noservice
+
+# Remove OSC working dir
+popd
+rm -r "$LOCAL_BRANCH_FOLDER"

packages/obs-setup.sh

@@ -7,6 +7,8 @@
 # * OBS_USER: username
 # * OBS_PASSWORD: password
 
+set -eu
+
 # Configure OSC
 cat > ~/.oscrc <<EOF
 [general]

processes/crystal-release.md

@@ -1,44 +1,69 @@
 # Crystal release process checklist
 
-1. Submit a Freeze/Release PR
+## Release preparation
+
+1. Announce expected release date and time span for feature freeze
+   * Feature freeze is about two weeks before release
+2. Start preparing changelog and release post
+3. Start feature freeze period
+   * Either no merging of features into `master` or split off release branch for backporting bugfixes.
+4. Publish release PR draft
    * It should contain the expected date of the release (~two weeks after the PR is issued).
-   * It should be populated with updates to CHANGELOG.md and VERSION.
-2. Verify Maintenance CI in the parent commit of the changelog succeed
-   * This mean either the commit was the one used for nightly build or,
-   * A branch with `release/` prefix or `ci` word was pushed
-3. Check the artifacts pass the ecosystem test
-   * Grab latest copy of https://github.com/crystal-lang/test-ecosystem
-   * Override branches of projects if needed at ./scripts/10-clone-repos.sh
-   * Set `./crystal-versions.env` with the URL of the artifacts and with the docker image
-   * Use the different make targets described in the `./README`
-4. Merge the Changelog PR
-5. Tag & annotate the commit with the changelog using `<M.m.p>` pattern as {version} (as a pre-release directly in GH?)
-6. Wait for the build
-7. Smoke test with test-ecosystem
-8. Push changes to OBS for building linux packages
-   1. Checkout https://github.com/crystal-lang/distribution-scripts
-   2. Grab the urls (or local path) of the artifacts from circle-ci
-      * `crystal-*-linux-x86_64.tar.gz`
-      * `crystal-*-linux-i686.tar.gz`
-      * `crystal-*-docs.tar.gz`
-   3. Follow [./packages/README.md](../packages/README.md) in distribution-scripts
-      1. The following steps should run in an openSUSE distribution with osc installed (https://github.com/crystal-lang/osc-docker)
-      1. Configure build.opensuse.org credentials in ~/.oscrc
-      1. `osc branchco devel:languages:crystal crystal`
-      1. `cd home\:$OBS_USERNAME\:branches\:devel\:languages\:crystal/crystal`
-      1. Copy the updated build artifact tarballs in place
-      1. Update version in package specifications:
-         ```bash
-         VERSION=1.1.0 # or whatever version you’re releasing
-         sed -i -e "s/^Version: .*/Version: ${VERSION}-1/" *.dsc
-         sed -i -e "s/^Version: .*/Version: ${VERSION}/" *.spec
-         ```
-      1. Verify the changes with `osc diff`
-      1. Update changelog: `osc vc -m “Update devel:languages:crystal to $VERSION`
-      1. Commit changes: `osc commit -m “Update devel:languages:crystal to $VERSION`
-      1. Now OBS builds the packages. It’s best to continue in the browser now: https://build.opensuse.org/project/show/home:$OBS_USER:branches:devel:langauges:crystal/crystal
-      1. Wait for all jobs to finish and when everything is green, create a submit request against the original package (“Submit package” link in the menu bar on the package in your branch)
-9. Build and publish docker images
+   * It should be populated with updates to `CHANGELOG.md` and `VERSION`.
+
+## Release process
+
+### Source release
+
+1. Finalize the release PR
+   * Make sure all changes are mentioned in the changelog
+   * Check release date
+   * Un-draft the PR
+2. Split off release branch (`release/x.y`)
+3. Verify Maintenance CI workflow succeeds on the HEAD of the release branch
+4. Smoke test with [test-ecosystem](https://github.com/crystal-lang/test-ecosystem)
+   * Run [*Test Crystal & Shards Workflow](https://github.com/crystal-lang/test-ecosystem/actions/workflows/test-crystal-shards.yml) with the release branch as `crystal_branch`.
+5. Merge the release PR
+6. Tag & annotate the commit with the changelog using `<M.m.p>` pattern as {version} (as a pre-release directly in GH?)
+7. Publish Github release
+   1. Copy the changelog section as description
+   1. Binaries are added later
+8. Publish docs
+   1. Have s3cmd installed and AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env variables defined
+   1. `$ source /Volumes/Secured/secure/manas-aws-key-crystal-dist.sh`
+   1. Checkout https://github.com/crystal-lang/crystal-dist
+   1. Download `*-docs.tar.gz` from circle-ci artifacts
+   1. `$ ./dist.sh push-docs {version} path/to/crystal-{version}-docs.tar.gz`
+   1. `$ ./dist.sh update-docs-versions {crystal_working_copy}`
+   1. `$ ./dist.sh redirect-docs {version}`
+
+### Binary releases
+
+1. Wait for the release build in circle CI
+2. Smoke test with test-ecosystem (again)
+3. Attach build artifacts to Github release
+   * `crystal-*-darwin-*.tar.gz`
+   * `crystal-*-linux-*.tar.gz`
+   * `crystal-*.pkg`
+   * `crystal-*-docs.tar.gz`
+4. Push changes to OBS for building linux packages
+   1. Checkout https://github.com/crystal-lang/distribution-scripts and go to [`./packages`](../packages)
+   2. Configure build.opensuse.org credentials in environment variables:
+      * `export OBS_USER=`
+      * `export OBS_PASSWORD=`
+   3. Run [`./obs-release.sh devel:languages:crystal crystal $VERSION`](../packages/obs-release.sh)
+      * Uses the docker image `crystallang/osc` to run the CLI client for OBS.
+      * The script creates a branch in you home project, updates the version and pushes it back to OBS.
+      * You can also run the commands from that file manually and check build locally with
+         * `osc build xUbuntu_20.04 x86_64`
+         * `osc build Fedora_Rawhide x86_64`
+   4. Now OBS builds the packages. It’s best to follow the build status in the browser:
+      1. `open https://build.opensuse.org/project/show/home:$OBS_USER:branches:devel:langauges:crystal/crystal`
+      1. Wait for all package build jobs to finish and succeed
+   5. Verify package installation
+      * `OBS_PROJECT=home:$OBS_USER:branches:devel:languages:crystal bats test`
+   6. When everything is green, create a submit request against the original package (*Submit package* link in the menu bar on the package in your branch)
+5. Build and publish docker images
    1. Checkout https://github.com/crystal-lang/crystal-dist
    2. `$ ./dist.sh build-docker {version}`
    3. `$ ./dist.sh push-docker {version}`
@@ -52,46 +77,38 @@
       * `$ docker push crystallang/crystal:{version}-alpine-build`
       * `$ docker tag crystallang/crystal:{version}-alpine crystallang/crystal:latest-alpine`
       * `$ docker push crystallang/crystal:latest-alpine`
-10. Publish docs
-    1. Have s3cmd installed and AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env variables defined
-    1. `$ source /Volumes/Secured/secure/manas-aws-key-crystal-dist.sh`
-    1. Checkout https://github.com/crystal-lang/crystal-dist
-    1. Download `*-docs.tar.gz` from circle-ci artifacts
-    1. `$ ./dist.sh push-docs {version} path/to/crystal-{version}-docs.tar.gz`
-    1. `$ ./dist.sh update-docs-versions {crystal_working_copy}`
-    1. `$ ./dist.sh redirect-docs {version}`
-11. Publish snap package
-    1. You need to logged in via `$ snapcraft login`
-    1. Recent tagged release is published directly to edge channel. The CI logs the snap revision number. Otherwise the .snap file is in the artifacts.
-    1. Check the current status to find the revision of the tagged release otherwise:
-    1. `$ snapcraft status crystal`
-    1. `$ snapcraft release crystal <revision-number> beta`
-    1. `$ snapcraft release crystal <revision-number> stable`
-12. Build Github release
-    1. Copy the changelog as release notes in the tag (if not done before)
-    1. Upload the signed binaries, *.tar.gz and *.pkg artifacts
-    1. Signed binaries are in bintray/build/signed
-    1. Publish the Github release
-13. Submit a PR to update the homebrew formula in https://github.com/Homebrew/homebrew-core/blob/master/Formula/crystal.rb .
-    1. Update the previous and new version (with their respective hashes).
-    1. Try locally `$ brew install --build-from-source <source of formula>`
-    1. Create PR
-14. Update crystal-website
-    1. Post a release notes in the website
-    1. Update information of `latest_release` in https://github.com/crystal-lang/crystal-website/blob/master/_config.yml
-    1. Update link to blog in https://github.com/crystal-lang/crystal-website/blob/5905e5644fcb4863976d0ca69856edbd74091370/index.html#L26
-15. Post announcement in https://forum.crystal-lang.org/c/news/official
-16. Tweet about the release
-17. Post in Reddit
-18. Update crystal repo to use released version
+6. Publish snap package
+   1. You need to logged in via `$ snapcraft login`
+   1. Recent tagged release is published directly to edge channel. The CI logs the snap revision number. Otherwise the .snap file is in the artifacts.
+   1. Check the current status to find the revision of the tagged release otherwise:
+   1. `$ snapcraft status crystal`
+   1. `$ snapcraft release crystal <revision-number> beta`
+   1. `$ snapcraft release crystal <revision-number> stable`
+7. Submit a PR to update the homebrew formula in https://github.com/Homebrew/homebrew-core/blob/master/Formula/crystal.rb .
+   1. Update the previous and new version (with their respective hashes).
+   1. Try locally `$ brew install --build-from-source <source of formula>`
+   1. Create PR
+
+### Release announcements
+1. Update crystal-website
+   1. Post a release notes in the website
+   1. Update information of `latest_release` in https://github.com/crystal-lang/crystal-website/blob/master/_config.yml
+   1. Update link to blog in https://github.com/crystal-lang/crystal-website/blob/5905e5644fcb4863976d0ca69856edbd74091370/index.html#L26
+2. Post announcement in https://forum.crystal-lang.org/c/news/official
+3. Tweet about the release
+4. Post in Reddit
+5. Update https://github.com/crystal-lang/crystal-book/blob/master/crystal-version.txt
+
+## Post-release
+1. Update crystal `master` branch to use released version
    * Edit PREVIOUS_CRYSTAL_BASE_URL in `.circleci/config.yml`
    * Edit DOCKER_TEST_PREFIX in `bin/ci`
    * Edit `prepare_build` on_osx download package and folder
    * Edit ` .github/workflows/win.yml` to point to docker image
    * Edit `shell.nix` `latestCrystalBinary` using  `nix-prefetch-url --unpack <url>`
-19. Increment VERSION file to the next minor and -dev suffix
-20. Perform uncomment/todos left in the repo
-21. Update carc.in / play.crystal-lang.org (by jhass)
+2. Increment VERSION file to the next minor and -dev suffix
+3. Perform uncomment/todos left in the repo
+4. Update carc.in / play.crystal-lang.org (by jhass)
     Note: This is done after Arch repos are updated
 
 ## Observable Helper