Onnxrt ep config

[manickavela29]

No description provided.

[manickavela29]

Hi,
I haven't tried with any Offlinemodel, so I am extending the config support to only for OnlineModel,
but easily it can be extended for OfflineModel.
Let me know if you have any better suggestions for maintaining the config.

session.cc files also has several other models (vad,etc), since these are minimal models, they are left as they are
if the support for trt/cuda backends can be extended for them.
let me know

for pybind interface, still adding in changes

[csukuangfj]

Hi, I haven't tried with any Offlinemodel, so I am extending the config support to only for OnlineModel, but easily it can be extended for OfflineModel. Let me know if you have any better suggestions for maintaining the config.

session.cc files also has several other models (vad,etc), since these are minimal models, they are left as they are if the support for trt/cuda backends can be extended for them. let me know

for pybind interface, still adding in changes

Thanks! That sounds good to me.

[csukuangfj]

Please have a look at errors reported by clang-tidy
https://github.com/k2-fsa/sherpa-onnx/actions/runs/9665507110/job/26662771827

and the style check by cpplint.

https://github.com/k2-fsa/sherpa-onnx/actions/runs/9665507136

(Note: You can run these two tools locally.)

[manickavela29]

I wanted to keep provider config unique for each models, this would give good control for CUDA and TRT configs,
But if developers give different devices(outside of cuda/trt) is a complex scenario and it must be handled, so I remove those patches midway.

Most of the fixes I had also went away with those patches thats why several bugs

[manickavela29]

Please have a look at errors reported by clang-tidy https://github.com/k2-fsa/sherpa-onnx/actions/runs/9665507110/job/26662771827

and the style check by cpplint.

https://github.com/k2-fsa/sherpa-onnx/actions/runs/9665507136

(Note: You can run these two tools locally.)

yes, have been using for cpplint, had a hard time with exit 🙂
will try with clangtiddy

[csukuangfj]

By the way, you can use

pip install clang-tidy

[manickavela29]

Python api is having some issues, can you direct me to particular workflowfile to build and test with it

Have you fixed it? Sorry for the late reply.

yes this is fixed, waiting for some build tests to see

[csukuangfj]

what is the cmd to run this?

Just start your terminal, activate your python virtual environment, and run

pip install clang-tidy

and then you can run

cd /path/to/sherpa-onnx
mkdir build
cd build
cmake -DSHERPA_ONNX_ENABLE_PYTHON=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..
make check

[manickavela29]

what is the cmd to run this?

Just start your terminal, activate your python virtual environment, and run

pip install clang-tidy

and then you can run

cd /path/to/sherpa-onnx
mkdir build
cd build
cmake -DSHERPA_ONNX_ENABLE_PYTHON=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..
make check

Yes got it, took it from the workflow file 🙌

[manickavela29]

Any suggestion for datatype

/home/runner/work/sherpa-onnx/sherpa-onnx/sherpa-onnx/csrc/provider-config.h:30:36: warning: implicit conversion from 'long' to 'int32_t' (aka 'int') changes value from 2147483648 to -2147483648 [-Wconstant-conversion]
int32_t trt_max_workspace_size = 2147483648;
^~~~~~~~~~

[csukuangfj]

Any suggestion for datatype

/home/runner/work/sherpa-onnx/sherpa-onnx/sherpa-onnx/csrc/provider-config.h:30:36: warning: implicit conversion from 'long' to 'int32_t' (aka 'int') changes value from 2147483648 to -2147483648 [-Wconstant-conversion]
int32_t trt_max_workspace_size = 2147483648;
^~~~~~~~~~

For tensorrt related variables, you can use uint32_t.

[manickavela29]

Most of the workflows are failing before build step itself, I think there is a pip dependency issue or something
Anyway it is good for review as of now

for jni, I have updated at one place, but I am not sure if it will fix it will completely
I am not very familiar with other interfaces, maybe you can update them accordingly.

I think it is good for review

[csukuangfj]

By the way, you can ignore the failed websocket server tests.

[manickavela29]

Handled suggested review

[manickavela29]

Hi @csukuangfj,
Let me know if there is any more suggestions

[csukuangfj]

By the way, could you fix the python tests. Please see

https://github.com/k2-fsa/sherpa-onnx/actions/runs/9689759305/job/26738528427?pr=992

[manickavela29]

I ran the python tests in debug mode and got this error

• python3 ./python-api-examples/offline-telespeech-ctc-decode-files.py
  Traceback (most recent call last):
  File "/mnt/efs/manickavela/asr_work/trt/sherpa-onnx/./python-api-examples/offline-telespeech-ctc-decode-files.py", line 16, in
  import sherpa_onnx
  File "/mnt/efs/manickavela/miniconda3/envs/icefallmk/lib/python3.9/site-packages/sherpa_onnx/init.py", line 1, in
  from _sherpa_onnx import (
  ImportError: arg(): could not convert default argument 'trt_config: sherpa_onnx::TensorrtConfig' in method '<class '_sherpa_onnx.ProviderConfig'>.init' in
  to a Python object (type not registered yet?)

it would be great to have some hints for fixing this
I tried to take reference from vad /online model config implementation and attempted but it didn't help.
I will dump a patch with my experiments and later clear it