Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MatMulNBits shape infer to SymbolicShapeInference #21246

Merged
merged 3 commits into from
Jul 5, 2024

Conversation

fajin-corp
Copy link
Contributor

@fajin-corp fajin-corp commented Jul 4, 2024

Description

Support MatMulNBits shape infer in SymbolicShapeInference

MatMulNBits's B input is rank-2, so implicit merge does not apply.

Motivation and Context

Issue with performing shape inference using symbolic_shape_infer.py with Phi-3 ONNX Models · Issue #21194 · microsoft/onnxruntime (github.com)

@tianleiwu
Copy link
Contributor

@fajin-corp, The python format pipeline failed. Please fix it by running lintrunner at the root like

pip install -r requirements-lintrunner.txt
pip install lintrunner
lintrunner init
lintrunner -a

@kunal-vaishnavi, please take a look.

@kunal-vaishnavi
Copy link
Contributor

With this PR's changes, can the symbolic shape inference tool now successfully run on the uploaded Phi-3 ONNX models?

@fajin-corp
Copy link
Contributor Author

fajin-corp commented Jul 5, 2024

With this PR's changes, can the symbolic shape inference tool now successfully run on the uploaded Phi-3 ONNX models?

I can shape infer the phi-3 int4 model with MatMulNBits in it.

@fajin-corp
Copy link
Contributor Author

/azp run Windows CPU CI Pipeline

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@fajin-corp fajin-corp merged commit 83e0c6b into main Jul 5, 2024
100 checks passed
@fajin-corp fajin-corp deleted the fajin/matmulnbitsshapeinfer branch July 5, 2024 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants