fix #579 Auto-merge between branches

[pxLi]

Signed-off-by: Peixin Li pxli@nyu.edu

ref: #579

[pxLi]

build

[pxLi]

build

[tgravescs]

How do we know if auto merge failed? Does it send to slack or anything?

[jlowe]

How do we know if auto merge failed?

The auto-merge PR remains open instead of being closed, and a comment is posted to the PR stating it needs manual intervention to be merged.

[jlowe]

I took a look at the script that does the merge, and there's an important part missing. The PR posted to do the automatic merge does not have CI run on it. Even if there are no merge conflicts, the PR should not be automatically merged unless CI passes on the merged result.

[pxLi]

How do we know if auto merge failed? Does it send to slack or anything?

There will be an email sent to maintainers.

sth like,

Run failed for branch-0.2 (5e0d70b)
Repository: pxLi/spark-rapids
Workflow: auto-merge
Duration: 30.0 seconds
Finished: 2020-08-19 05:08:32 UTC

View results

Jobs:
automerge failed (0 annotations)

[pxLi]

I took a look at the script that does the merge, and there's an important part missing. The PR posted to do the automatic merge does not have CI run on it. Even if there are no merge conflicts, the PR should not be automatically merged unless CI passes on the merged result.

I think we add the auto-merge here is because we haven't set branch-0.3 as out default branch. So I am assuming we won't do active development on that branch until we switch that as default, and we can terminate this auto-merge workflow at that moment. But I agree that we can setup probably an independent nightly jenkins job to do the CI thing.

FYI cuDf does the same thing w/o CI pipeline rapidsai/cudf#6047.

Let me know if you have any other concern :)

[pxLi]

build

[jlowe]

I think we add the auto-merge here is because we haven't set branch-0.3 as out default branch.

Which branch is default is not relevant. The point of the auto-merge is to cover the time when we are moving from one development branch to another as part of making a release. That migration is not an instantaneous, hard cutover. Bug fixes continue to pour into the older branch, while new features need to go into the new branch to avoid destabilizing the older branch just before the release. It will often be days or even weeks where we will need CI on both branches. The point of the auto-merge is to automatically pull in bugfixes checked in on the older branch into the newer branch so they aren't lost when moving to the later releases.

I'd really like pre-merge CI to run on the auto-merge branch if possible, but a nightly CI on the newer branch is the next best thing. We need something to detect in a timely manner when the auto-merge breaks an existing test (or even the build!).

[pxLi]

I think we add the auto-merge here is because we haven't set branch-0.3 as out default branch.

Which branch is default is not relevant. The point of the auto-merge is to cover the time when we are moving from one development branch to another as part of making a release. That migration is not an instantaneous, hard cutover. Bug fixes continue to pour into the older branch, while new features need to go into the new branch to avoid destabilizing the older branch just before the release. It will often be days or even weeks where we will need CI on both branches. The point of the auto-merge is to automatically pull in bugfixes checked in on the older branch into the newer branch so they aren't lost when moving to the later releases.

I'd really like pre-merge CI to run on the auto-merge branch if possible, but a nightly CI on the newer branch is the next best thing. We need something to detect in a timely manner when the auto-merge breaks an existing test (or even the build!).

yep, I totally agree that we should have some detection for auto-merge stuff on the newer branch.

pre-merge checks will make the process less 'auto', since the new commits on older branch could get merged at the same time. Then it could cost multiple pre-merge CI runs to get that auto-merge PR passed checks, which could block forever in the worst case 🤦 (we could checkout a tmp branch from older branch every time to avoid that, but could bring more problems, sounds janky though)

I can set up a nightly pipeline for branch-03 later, and people will get notifications from slack just like our other CI jobs. Let me know if that sound reasonable to you 😸

[pxLi]

build

[pxLi]

build

[pxLi]

build

[pxLi]

build

[pxLi]

build

[pxLi]

build

[pxLi]

build