Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase resource usage for scheduled jobs #7811

Closed
wants to merge 3 commits into from
Closed

Conversation

kgpai
Copy link
Contributor

@kgpai kgpai commented Nov 30, 2023

Scheduled fuzzer jobs are currently ooming, https://github.com/facebookincubator/velox/actions/runs/7041471735/job/19164078488 .
Run them on larger instances.

Copy link

netlify bot commented Nov 30, 2023

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 8b1f8cd
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/6568e6c10c5532000826cb9e

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 30, 2023
@facebook-github-bot
Copy link
Contributor

@kgpai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Copy link
Collaborator

@assignUser assignUser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (assuming oss team has created the 16-core runners), I will look into mitigating the oom issues!

@facebook-github-bot
Copy link
Contributor

@kgpai merged this pull request in b3ed52e.

Copy link

Conbench analyzed the 1 benchmark run on commit b3ed52e1.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

facebook-github-bot pushed a commit that referenced this pull request Dec 8, 2023
Summary:
This PR tackles several issues with the scheduled fuzzer workflow:
- workaround compilation being OOMkilled by using only 8 threads and using `mold` as a linker
  - this is a stopgap measure only, we should really look at our build times/footprint and see how we can optimize that as a 45 minute build on an 8 core machine is pretty massive.
- only compile once (saves ~ 1.5h of CI time)
- use actions/cache to save the ccache to further improve build times
- add `workflow_dispatch` trigger to enable ad hoc runs of the fuzzer jobs
- add `pull_request` trigger to enable testing of the workflow in PRs (like this one ^^)
  - we could easily move the remaining CCI fuzzer job over to this workflow as well

Reverts #7811 to use 8-core machines again. Supersedes #7810

Pull Request resolved: #7842

Reviewed By: mbasmanova

Differential Revision: D51960006

Pulled By: kgpai

fbshipit-source-id: 0f493f80d5a87cca20bd53f1074a5f4622ee02b5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants