sync docs on tags

[robbymilo]

PR Description

• updates the publish action to trigger on tags
  • the test step still runs on push to the main branch
  • the sync step only runs on a tag event
  • the target_folder (website dir) uses the tag name
• also fixes the volume path in the test step

Todo:

• sync main branch to next dir

Needs https://github.com/grafana/website/pull/8124

[rfratto]

This is great!

One question comes to mind: how does this handle non-stable releases or untraditional release orders?

There are two scenarios that I can think of here:

1. What happens if we do a release candidate? (i.e., tag v0.24.0-rc.0) Should we exempt release candidates from this action?

2. What happens if we do a patch release for a previous minor version? (i.e., the latest release is v0.24.0 but we create a v0.23.1 to fix a critical bug in the previous minor version) Should we extend this action to do some kind of ordering check to only run for new minor releases?

[mdisibio]

Hi I am exploring similar automation for Tempo and have ran into the same questions and here is how I am planning to address them. I have a barebones github action which converts a branch or tag name ("github_ref_name") into the target folder vMajor.Minor, or also main->next. Then here is an example of a GH workflow to call it and use its output.

1. I was planning on excluding release candidates and other strays tags with a regex on the GH trigger itself.

2. If we build a patch we do want to sync the docs again, so the behavior of the action works as written is ok. It will overlay new docs from the x.y.z tag onto the existing vx.y docs folder. This will also trigger for any commit to the /docs/ folder on a fixed release branch, which handles the sync for non-build related changes that can occur whenever (i.e. fixing typos, verbiage tweaks, release notes, etc).

[GrafanaWriter]

Thank you

[tpaschalis]

Hey folks! Sorry for not following through with this last week. I agree with Marty's thoughts on both points.

We can use the fine-grained trigger to only trigger on specific tags and release branches to exempt release candidates, betas etc. Also, I agree it would be okay to re-sync the docs in cases of patch releases or new commits to release branches, so no manual ordering is needed.

@mdisibio I have 0 knowledge about Typescript, so initially to get the same results as you, I used some simple bash, regex+capture groups to extract major.minor from the triggered branch or tag.

if  [[ $input =~ ^v([0-9]+)\.([0-9]+)\.[0-9]+$ ]]; then
    echo "v${BASH_REMATCH[1]}.${BASH_REMATCH[2]}"
elif [[ $input =~ ^release-([0-9]+)\.([0-9]+)\.[0-9]+$ ]]; then
    echo "v${BASH_REMATCH[1]}.${BASH_REMATCH[2]}"
else
    exit 1
fi

I think we might be able to use this inline as a step in the GH action. Any preference on which of the two we should start experimenting with?

[mdisibio]

Great to hear, I think we are homing in on a good solution. I had also started with some simple bash scripting, but found it cumbersome to pass output to the next step (also my bash was not as good as yours). I think the github action will be simpler overall when considering how many repos could make use of this and any other changes we'd like in the future, but I am interested to see a full bash workflow too if you are. Oh I also updated the typescript to use semver/coerce which I was not aware of. Thanks @robbymilo

[tpaschalis]

You're kinda right, I wouldn't want to push the burden of adding hard-to-read commands on every project that wants to use it.

I'll toy around with the bash workflow on a branch of my own and ping you, but we can start by using your solution here.

[tpaschalis]

@mdisibio @robbymilo I took the liberty of separating the workflow in two Github Actions

1. publish-next.yml only runs on pushes to main, and always syncs to the /next directory
2. publish-versioned.yml runs on pushes to vX.Y.Z tags (not release candidates, betas etc), and release-* branches. It uses Marty's typescript snippet to extract the value and push to the correct /vX.Y directory.

I'll test some more in my own fork, but it seems to be a good start.

EDIT: What do y'all think? ^^

[mdisibio]

What do y'all think? ^^

Since the typescript snippet handles main branch, seems like a project preference for one workflow or two. No strong feelings either way.

[tpaschalis]

@mdisibio
Oh, so that's what this snippet does? If the branch is main, then it sets the output to next?

Still not sure what the ! is for on setTarget(knownRefs.get(ref)!), but yeah, we could have them as one!

[mdisibio]

@tpaschalis Yep by checking the string-string map at the top.

Still not sure what the ! is for on setTarget(knownRefs.get(ref)!), but yeah, we could have them as one!

The map .get(key) func returns a nullable string, but setTarget takes non-nullable. The ! operator removes nullability from the value. In this case it is safe because we already checked that the value exists. I don't use Typescript regularly, so there might be a more readable way.

[tpaschalis]

Alright, this seems to work 🎉

Pushing a new commit on main
Publishing docs/user to https://***@github.com/grafana/website.git:master/content/docs/agent/next

Pushing a new tag v1.2.99-rc1, no job was triggered

Pushing a new tag v1.2.99
Publishing docs/user to https://***@github.com/grafana/website.git:master/content/docs/agent/v1.2

Pushing a commit on the branch release-1.5.9
Publishing docs/user to https://***@github.com/grafana/website.git:master/content/docs/agent/v1.5

Pushing a commit on the branch release-1.6
Publishing docs/user to https://***@github.com/grafana/website.git:master/content/docs/agent/v1.6

[jdbaldry]

Below is me thinking out loud about the interaction of syncing on both release branches and on tags. I would love to know if my understanding matches with everyone here.

I imagine we expect tags to be immutable and would represent an update to the docs expected only when a new patch release is made. For subsequent docs changes, we would expect them to be cherry-picked into the corresponding release branch. Patch releases for previous versions would be branched from the previous patch release.

With those assumptions, I think this behaves as we want and ensures we never sync inconsistent documentation for released versions.

[tpaschalis]

@jdbaldry yes your assumptions sound about right!

We'll probably take this for a test drive in our Agent release this week, we'd be happy if other projects were able to use it as well!

[rfratto]

We need to determine how we intend to update documentation for versioned releases. I would suggest that we only sync documentation for a release when vX.Y.0 is tagged, but not any other patch release.

This means that documentation improvements can be immediately backported into a release by a PR to the docs folder.

If we leave this as-is, that means:

• You cannot manually backport changes to a release folder because it would get overwritten if we did a patch release
• You have to wait for a patch release to be created for updated release docs to get re-synced (if a patch release ever gets created at all)
• Backporting changes to multiple previous releases is more difficult, as you have to do it across one PR per release branch

[mdisibio]

Leaving some thoughts here but I may be missing some context that is specific to the Agent repo and workflow, so this is more from a Tempo point of view.

The tags and branch filters can independently trigger, so I think the workflow will handle the cases mentioned.

You cannot manually backport changes to a release folder because it would get overwritten if we did a patch release

It's expected to build the next patch off of the existing release branch, and any docs updates would have already been committed there. Could you give an example of what might get overwritten?

You have to wait for a patch release to be created for updated release docs to get re-synced (if a patch release ever gets created at all)

A commit to the release branch should resync immediately, no tag or release needed.

Backporting changes to multiple previous releases is more difficult, as you have to do it across one PR per release branch

This is true and was not a design goal. I'm not familiar with this use case, or sure if it's common enough to address. Since the website documentation is forked into subfolders, I'm not sure what automation would look like to sync them all off of 1 PR. What did you have in mind?

[rfratto]

It's expected to build the next patch off of the existing release branch, and any docs updates would have already been committed there. Could you give an example of what might get overwritten?

Right, I'm saying if copy changes are done to grafana/website so they appear immediately, and not synced to grafana/agent. The next patch release tag will re-sync the folder overriding any manual changes done.

1. Release v0.100.0 gets cut, v0.100 docs synced to grafana/website
2. Copy changes manually made to grafana/website/docs/agent/v0.100
3. Release v0.100.1 gets cut, v0.100 docs re-synced to grafana/website
4. Previous copy changes get lost because they were not made to grafana/agent

A commit to the release branch should resync immediately, no tag or release needed.

That should only be true after we cut a stable release, which won't be true for every release branch (unfortunately). We create a single release-X.Y branch to cover the entire X.Y.Z release "series," including release candidates.

This is true and was not a design goal. I'm not familiar with this use case, or sure if it's common enough to address. Since the website documentation is forked into subfolders, I'm not sure what automation would look like to sync them all off of 1 PR. What did you have in mind?

My understanding is that documentation writers will manually backport changes to grafana/website. With the current flow, they're able to backport to multiple previous versions at once (since it's one giant directory tree). If the source of truth becomes repositories, then it becomes slightly more tedious.

My understanding could be wrong, or it could also be that nobody would mind this change.

[rfratto]

FWIW, we document our current release process here.

[mdisibio]

copy changes are done to grafana/website

Ok yes, agree with everything you've said under that scenario. My understanding is that kind of activity is rare, and will be moreso after this automation. We do want the upstream repositories to be the source of truth.

backport to multiple previous versions at once

Agree, that scenario would not be supported. I have not heard it mentioned so thinking it's a worthwhile tradeoff for the increased automation in general. Definitely for Tempo and GET.

single release-X.Y branch

We do the same. I think I understand part of your concern in that under this setup the documentation would get to the website "early", before the final stable candidate is pushed. However, the thoughts are that it's not a blocker for a few reasons: (1) the time between starting a release (creating the branch) and the stable .0 tag is relatively short (a few days) (2) we can devise a tweak to the workflow later if needed (already have some rough ideas).

won't be true for every release branch

Got it, for Tempo that is not an issue. I think we have a good base to build on here, but expect every project will have some customization. Syncing on both tags and branch commits is already quite heavy-handed and more just to start with a generic solution and experimentation.

[GrafanaWriter]

My understanding is that documentation writers will manually backport changes to grafana/website. With the current flow, they're able to backport to multiple previous versions at once (since it's one giant directory tree). If the source of truth becomes repositories, then it becomes slightly more tedious.

This isn't the preferred approach. The website should reflect the technical documentation stored in the product/project repo, with the website automatically reflecting that change. Thus, backporting should happen only in the product/project repo.

More detail: if we update the website, we have to make a duplicate PR in the respective product/project repo. Duplicate work is not only inefficient, but also prone to error.

[rfratto]

Thanks for explaining!

[GrafanaWriter]

Backporting is relatively common in some products, not as frequent in others. We'd like to enable a consistent approach. And our aim is to not backport more than two minor releases back, similar to the backporting policy for security fixes.

[GrafanaWriter]

Agree, that scenario would not be supported. I have not heard it mentioned so thinking it's a worthwhile tradeoff for the increased automation in general. Definitely for Tempo and GET.

I agree

[GrafanaWriter]

I think we've teased out some good issues here. @jdbaldry may also want to comment, as he's focused now on the docs build pipeline with the goal of creating more consistency and sharing practices.

Thank you for the dialog on this - I think it's definitely a step forward

[mattdurham]

Waiting to merge this until we have reviewed Agent documentation

[github-actions]

This PR has been automatically marked as stale because it has not had any activity in the past 30 days.
The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed in 7 days if there is no new activity.
Thank you for your contributions!