Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: disable jobs that failed > 50% on nightly CI recently - part 1 #9765

Merged
merged 4 commits into from
Jun 4, 2024
Merged

CI: disable jobs that failed > 50% on nightly CI recently - part 1 #9765

merged 4 commits into from
Jun 4, 2024

Conversation

wainersm
Copy link
Contributor

I've made a python script to collect some statistics of the nightly CI jobs. This PR disables the test jobs that failed > 50% on the last 10 nights.

While I think that this script has bugs (e.g. I notice the filter regex isn't correct), I do believe that for the jobs I'm disabling here, the numbers are correct. I've observed those jobs for a while now and I only wanted an excuse :)

See the script and results here: #9506 (comment)

@wainersm
CI: disable run-kata-monitor-tests / run-monitor (qemu, containerd) job
ac4d48a 
This job has failed more than 50% on nightly CI. Remove it from the list of
execution until we don't have a fix.

Issue: 9761
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
@wainersm
CI: disable run-basic-amd64-tests / run-tracing jobs
9154ce9 
These jobs have failed more than 50% on nightly CI. Remove them from the list of
execution until we don't have a fix.

Issue: 9763
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
@wainersm
CI: disable run-basic-amd64-tests / run-vfio (clh) job
5f5274e 
The job has failed more than 50% on nightly CI. Remove it from the list of
execution until we don't have a fix.

Issue: 9764
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
@wainersm
github: add actionlint configuration file
c9f93fc 
Added configuration file with rules to exclude some self-hosted
runners from the linter warnings.

Related-with: kata-containers#9646
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
@wainersm wainersm added the area/ci Issues affecting the continuous integration label May 31, 2024
@katacontainersbot katacontainersbot added the size/small Small and simple task label May 31, 2024
stevenhorsman
stevenhorsman approved these changes Jun 3, 2024
View reviewed changes
Copy link
Member

@stevenhorsman stevenhorsman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I think we want to ensure that everyone is aware that some tests are being disabled now - maybe we could create a tailorbird project board and have the issues that are spun out tracked in there?

@wainersm
Copy link
Contributor Author

wainersm commented Jun 3, 2024

LGTM, but I think we want to ensure that everyone is aware that some tests are being disabled now - maybe we could create a tailorbird project board and have the issues that are spun out tracked in there?

That's a good idea. I don't have the superpowers to create one, mind to do that favor to me? :)

I will advertise that PR on our slack channels to ensure that devs are aware and can comment about those disables. So let's hold the merge a little bit.

@stevenhorsman
Copy link
Member

LGTM, but I think we want to ensure that everyone is aware that some tests are being disabled now - maybe we could create a tailorbird project board and have the issues that are spun out tracked in there?

That's a good idea. I don't have the superpowers to create one, mind to do that favor to me? :)

I've created https://github.com/orgs/kata-containers/projects/46/views/1, so hopefully that works for you?

@wainersm
Copy link
Contributor Author

wainersm commented Jun 3, 2024

LGTM, but I think we want to ensure that everyone is aware that some tests are being disabled now - maybe we could create a tailorbird project board and have the issues that are spun out tracked in there?

That's a good idea. I don't have the superpowers to create one, mind to do that favor to me? :)

I've created https://github.com/orgs/kata-containers/projects/46/views/1, so hopefully that works for you?

It does; thanks @stevenhorsman !

@fidencio
Copy link
Member

fidencio commented Jun 3, 2024

I've created https://github.com/orgs/kata-containers/projects/46/views/1, so hopefully that works for you?

I've made this public.

fidencio
fidencio approved these changes Jun 3, 2024
View reviewed changes
Copy link
Member

@fidencio fidencio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks @wainersm!

@stevenhorsman
Copy link
Member

I've made this public.

The real org owner superpower here!

@wainersm
Copy link
Contributor Author

wainersm commented Jun 4, 2024

LGTM, but I think we want to ensure that everyone is aware that some tests are being disabled now - maybe we could create a tailorbird project board and have the issues that are spun out tracked in there?

That's a good idea. I don't have the superpowers to create one, mind to do that favor to me? :)

I will advertise that PR on our slack channels to ensure that devs are aware and can comment about those disables. So let's hold the merge a little bit.

I sent a message to the kata-dev channel. Nobody is against this PR. So added the label so we can pass it through CI and merge.

@wainersm wainersm merged commit 2b8cdd9 into kata-containers:main Jun 4, 2024
301 of 309 checks passed
@likebreath likebreath mentioned this pull request Jun 11, 2024
Merged
@wainersm wainersm mentioned this pull request Jun 14, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fidencio fidencio fidencio approved these changes

@GabyCT GabyCT GabyCT approved these changes

@stevenhorsman stevenhorsman stevenhorsman approved these changes

Assignees
No one assigned
Labels
area/ci Issues affecting the continuous integration ok-to-test size/small Small and simple task
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@wainersm @stevenhorsman @fidencio @GabyCT @katacontainersbot