[Build] Build python interface for Onnxruntime-qnn on aarch64 Linux #21203
Comments
github-actions
bot
added
the
platform:windows
snnn
added
ep:QNN
|
the QNN docs don't mention aarch64 Linux support, but you're right, the QNN SDK does seem to have some aarch64--linux directories under lib onnxruntime/cmake/CMakeLists.txt Line 832 in 7e93cd7 to target one of the aarch64--linux , currently it assumes non x86_64 Linux is aarch64-android but that's not the case for you. |
|
Hi, thank you for the advise. so I did try to do the compiling on a x86_64 ubuntu machine, and edit the CMakeLists.txt as you suggested: and here is the command I use: The building process is all passed until the step |
|
Do you have any update on solving this issue? (compiling onnx_qnn on aarch linux) Our team is having the same problem with setting up the cmake file. By the way, does this onnx_qnn allow dynamic model input? |
|
are you all trying to cross compile or build natively on aarch64 Linux? |
|
For me, I tried to build natively on aarch64 Linux first but then realized that I can't setup the QNN SDK on that environment. |
what setup are you missing. i think you can try copying the qnn sdk installation directory (e.g. /path/to/qnn/2.23.0.240531) from an installation to different platform (e.g. Linux x64) to the aarch64 linux environment , make the cmake modification mentioned above and give it another try. |
|
yes, I did try that before and it seems to failed. Thank you for the advice though, I'll still try to work on that. |
what kind of failure do you get. we can reach out to qualcomm and ask for guidance. |
|
[edited] So I did some debugging and resolve the previous error, and the failure now is listed below: |
|
Okay, so I've been trying to figure out the dependencies and configurations, and finally I did build a python wheel with the following command: bash build.sh --build --arm64 --use_qnn --qnn_home=/path/to/qnn/2.22.6.240515/ --parallel --build_wheel --skip_submodule_sync --config ReleaseHowever, when I tried to follow the instruction and run a basic mobilenet_v3_small, the following error raised: The script I run is: import onnxruntime
import numpy as np
options = onnxruntime.SessionOptions()
# (Optional) Enable configuration that raises an exception if the model can't be
# run entirely on the QNN HTP backend.
# options.add_session_config_entry("session.disable_cpu_ep_fallback", "1")
# Create an ONNX Runtime session.
session = onnxruntime.InferenceSession("mobnet.qdq.onnx",
sess_options=options,
providers=["QNNExecutionProvider"],
provider_options=[{"backend_path": "./lib/libQnnHtp.so"}])
# Run the model with your input.
# TODO: Use numpy to load your actual input from a file or generate random input.
input0 = np.ones((1,3,224,224), dtype=np.float32)
while True:
result = session.run(None, {"input.1": input0})
# Print output.
print(result) |
|
Did you solve the issue with "QNN SetupBackend failed Failed to create device"? |
|
The error code On Andriod, you'll probably also need to reference <uses-library
android:name="libcdsprpc.so"
android:required="true"/>See: https://developer.qualcomm.com/sites/default/files/docs/snpe/dsp_runtime.html |
Describe the issue
So I'm using a development kit based on QCS 8550 SoC with an Ubuntu 22.04 on it, and trying to build onnxruntime-qnn on it for python interface.
I'm following the instruction here but was not able to set up the QNN SDK properly. It seems like Qualcomm has not officially supported QNN SDK for aarch64 linux, and thus we are not able to build ort locally on the machine with that instruction.
However, this system did contains the library files like libQnnDsp.so/libQnnHtp.so somewhere, and I can use them for inferencing by tools like qnn-net-run.
Is there any way that I can build the onnxruntime with qnn backend by cross-building from x86 machine or local build from source?
Urgency
The time line for us is pretty tight and we are looking forwards to a solution ASAP.
Target platform
Customized aarch64 Linux with Qualcomm SoC
Build script
Error / output
Visual Studio Version
No response
GCC / Compiler Version
No response
The text was updated successfully, but these errors were encountered: