Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document why master merge before the check is necessary #17

Closed
joao-paulo-parity opened this issue Dec 14, 2021 · 3 comments
Closed

Document why master merge before the check is necessary #17

joao-paulo-parity opened this issue Dec 14, 2021 · 3 comments

Comments

@joao-paulo-parity
Copy link
Contributor

joao-paulo-parity commented Dec 14, 2021

Initially discussed in #16 (comment) and more motivation given in #16 (comment).

This change might have caused a failure reported in the CI channel

not sure what's happening here: paritytech/polkadot#4500 but it looks like the cumulus companion is being built with a commit of polkadot which is not that of the polkadot PR

The Cumulus companion was cloned with SHA 042f3c4, but then more code was brought in from master which might have caused the failure. Regardless of this being the actual reason or not, there are two problems to consider

  1. Code is brought in from outside the branch without the developer being aware of it since this behavior is not documented nor motivated
  2. We lose the ability to reason about the companion's code in isolation since the code from master was brought in, whatever it was at that point in time (at the very least the SHA used for the merge should be printed)
@joao-paulo-parity
Copy link
Contributor Author

cc #16 (comment)

@joao-paulo-parity
Copy link
Contributor Author

joao-paulo-parity commented Dec 15, 2021

#16 (comment) hints at the source of the problem

You will try to compile with Polkadot master, but that will fail because you don't have the required Substrate changes in your branch

It all stems from the master code being merged into the PR branch at the start

https://github.com/paritytech/pipeline-scripts/blob/master/check_dependent_project.sh#L232-L233

Therefore all the companions also have to buy into this approach since each of them might also not have the "possibly breaking changes" necessary to keep up with the job's PR brought-in master code.

It is possible to sidestep the problem entirely by not merging master into any branch before doing the checks.

Pro: Avoids NonDeterminism ^ N for each PR in the chain due to the external code being brought in from master (possibly the cause of #18). By following suit, should a failure happen, one does not have to reason about what the master for each branch had at some arbitrary point in time; crucially neither the timestamp for this step nor the master's SHA are printed, so even if we keep the current approach, the logging of what is going on has to be improved.

Con: We'll give up on the "pre-merge" intention mentioned in the comments

# Merge master into our branch so that the compilation takes into account how the code is going to
# perform when the code for this pull request lands on the target branch (à la pre-merge pipelines).

IMO that is fine because pre-merge pipelines are implemented elsewhere (https://github.com/paritytech/ci_cd/issues/162).

joao-paulo-parity added a commit to joao-paulo-parity/pipeline-scripts that referenced this issue Dec 15, 2021
…m and simplification; pre-merge flows can be implemented on some other check

more context given in paritytech#17 (comment)
@joao-paulo-parity
Copy link
Contributor Author

After discussing the problem at length with @TriplEight (some of the topics should be moved to docs at some point), the upside of merging master seems worth it despite all the extra complication. By doing so we'll have better expectations of the integration not breaking after a PR lands on master.

Nondeterminism due to fetching + merging alien code into the branches throughout the whole dependency chain still makes the pipeline overall harder to reason about. We'll attempt to counteract that by providing better failure messages and raising awareness of how the job works step-by-step.

Implementation starts at dae5933.

joao-paulo-parity added a commit that referenced this issue Dec 17, 2021
…m and simplification; pre-merge flows can be implemented on some other check

more context given in #17 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant