Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor Python CUDA packaging pipeline to fix random hangs in building #19989

Merged
merged 32 commits into from
Mar 22, 2024
Merged

Refactor Python CUDA packaging pipeline to fix random hangs in building #19989

merged 32 commits into from
Mar 22, 2024

Conversation

mszhanyi
Copy link
Contributor

@mszhanyi mszhanyi commented Mar 20, 2024 •
edited
Loading

Description

  1. Move building on CPU machine.
  2. Optimize the pipeline
  3. Since there isn't official ONNX package for python 12, the python 12 test stage uses the packages built with ONNX source in build stage.

Motivation and Context

  1. Resolve the random hang in compilation
  2. Save a lot of GPU resources.

@mszhanyi mszhanyi requested a review from a team as a code owner March 20, 2024 06:56
@mszhanyi mszhanyi changed the title Refactor Python CUDA packaging pipeline Refactor Python CUDA packaging pipeline to fix random hangs in building Mar 20, 2024
@mszhanyi mszhanyi marked this pull request as draft March 20, 2024 14:05
@mszhanyi mszhanyi marked this pull request as ready for review March 20, 2024 15:28
@mszhanyi mszhanyi marked this pull request as draft March 20, 2024 15:29
@snnn
Copy link
Member

snnn commented Mar 20, 2024

You may consider also uploading ONNX's wheel file from the CPU stage.

@mszhanyi mszhanyi marked this pull request as ready for review March 21, 2024 15:33
snnn
snnn previously approved these changes Mar 21, 2024
View reviewed changes
Copy link
Member

@snnn snnn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@mszhanyi mszhanyi requested a review from snnn March 21, 2024 23:30
@mszhanyi mszhanyi merged commit cd6d3ae into main Mar 22, 2024
95 of 97 checks passed
@mszhanyi mszhanyi deleted the zhanyi/cudapackaing branch March 22, 2024 01:16
TedThemistokleous pushed a commit to TedThemistokleous/onnxruntime that referenced this pull request May 7, 2024
@mszhanyi
Refactor Python CUDA packaging pipeline to fix random hangs in buildi…
75044fc 
…ng (microsoft#19989)

### Description
1. Move building on CPU machine.
2. Optimize the pipeline
3. Since there isn't official ONNX package for python 12, the python 12
test stage uses the packages built with ONNX source in build stage.


### Motivation and Context
1. Resolve the random hang in compilation
4. Save a lot of GPU resources.

---------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@snnn snnn snnn approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mszhanyi @snnn