[Documentation] Setup the CUDA Environment is not detailed enough

[BrainSlugs83]

Describe the documentation issue

I'm following this guide, which links to this page.

The instructions on that latter page are not detailed enough.
I have "CUDA_PATH" set correctly, and it's working for other things that require CUDA 11.8.

But I'm still getting DLL LOAD ERRORS when I try to run the script -- but it's an ONNX DLL and not a CUDA one. And the DLL appears to actually exist. So, it's not really clear what the issue is to me.

onnxruntime_genai.onnxruntime_genai.OrtException: C:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\Mikey\miniconda3\envs\phi3\Lib\site-packages\onnxruntime_genai\onnxruntime_providers_cuda.dll"

Proof that the DLL Exists:

> dir C:\Users\Mikey\miniconda3\envs\phi3\Lib\site-packages\onnxruntime_genai\onnxruntime_providers_cuda.dll
 Volume in drive C is Windows
 Volume Serial Number is 784B-4CD0

 Directory of C:\Users\Mikey\miniconda3\envs\phi3\Lib\site-packages\onnxruntime_genai

06/27/2024  11:53 AM       410,648,608 onnxruntime_providers_cuda.dll
               1 File(s)    410,648,608 bytes
               0 Dir(s)  1,124,985,217,024 bytes free

Also proof of CUDA_PATH being set:

> dir "%CUDA_PATH%"
 Volume in drive C is Windows
 Volume Serial Number is 784B-4CD0

 Directory of C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8

10/20/2023  02:08 PM    <DIR>          .
10/20/2023  02:06 PM    <DIR>          ..
10/20/2023  02:08 PM    <DIR>          bin
10/20/2023  02:08 PM    <DIR>          compute-sanitizer
08/25/2022  03:33 PM            80,660 CUDA_Toolkit_Release_Notes.txt
08/25/2022  03:33 PM               160 DOCS
08/25/2022  03:33 PM            61,498 EULA.txt
10/20/2023  02:08 PM    <DIR>          extras
10/20/2023  02:08 PM    <DIR>          include
10/20/2023  02:08 PM    <DIR>          lib
10/20/2023  02:08 PM    <DIR>          libnvvp
10/20/2023  02:08 PM    <DIR>          nvml
10/20/2023  02:08 PM    <DIR>          nvvm
08/25/2022  03:33 PM               524 README
10/20/2023  02:08 PM    <DIR>          src
10/20/2023  02:08 PM    <DIR>          tools
09/29/2022  11:17 AM             2,608 version.json
               5 File(s)        145,450 bytes
              12 Dir(s)  1,124,984,737,792 bytes free

Page / URL

https://onnxruntime.ai/docs/genai/howto/setup-cuda-env.html#special-instructions-on-windows

[jywu-msft]

I think it's missing the steps to install CuDNN which is also a dependency
https://onnxruntime.ai/docs/install/#cuda-and-cudnn

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.