CUDA card not detected.

[evertvorster]

Hi there.
Firstly, thanks so much for the work you put into this.

I finally managed to install the nvidia-libs, and I'm pretty sure its working, as it passed the tests.

[evert@Evert 0.7.19]$ WINEPREFIX=/home/evert/.local/share/bottles/bottles/Insta360 ./bin/nvapi64-tests.exe

returns
All tests passed (94 assertions in 1 test case)

When running the GPU caps viewer, unfortunately it does not detect CUDA.

What is interesting to note is that OpenGL is running on my other gpu, which is an AMD.
One other thing I have tried is to install CUDA, and that fails too...
The error message says "Call not implemented"

Now the reason I would like to have CUDA acceleration is that I would like to run Insta360 Studio in a bottle, with CUDA acceleration. It runs fine as it is, but fails to detect the GPU for accelerated video decode and encode.

Here is a little more information on my laptop:

Operating System: Arch Linux 
KDE Plasma Version: 6.1.5
KDE Frameworks Version: 6.6.0
Qt Version: 6.7.2
Kernel Version: 6.10.8-arch1-2.1-g14 (64-bit)
Graphics Platform: Wayland
Processors: 32 × AMD Ryzen 9 7945HX3D with Radeon Graphics
Memory: 62.0 GiB of RAM
Graphics Processor: AMD Radeon 610M
Manufacturer: ASUSTeK COMPUTER INC.
Product Name: ROG Strix G733PYV_G733PYV
System Version: 1.0

It contains a secondary GPU, which is the mobile version of the nVidia RTX 4090.

[evert@Evert 0.7.19]$ nvidia-smi
Thu Sep 26 14:19:54 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.03              Driver Version: 560.35.03      CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090 ...    Off |   00000000:01:00.0  On |                  N/A |
| N/A   50C    P5             18W /  175W |      87MiB /  16376MiB |     42%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      1001      G   /usr/lib/Xorg                                   4MiB |
|    0   N/A  N/A      1233      G   /usr/bin/kwin_wayland                           4MiB |
+-----------------------------------------------------------------------------------------+

[evertvorster]

Oh, one other thing. The only time the test passes is if I use the system's version of Wine. With all the other latest versions of the runners the test program fails.

[SveSop]

I am sure the nvapi libs are working as intended, but it seems nvcuda is not. The nvapi64-tests.exe binary run a test against nvapi, and that is not the same as nvcuda.

If you could run gpu caps viewer with WINEDEBUG=+nvcuda (some debug setting in bottles i suppose you can add this), and see if something is listed.. or perhaps WINEDEBUG=+loaddll to see if the nvcuda lib is even loaded.

[evertvorster]

Thanks for giving me more to look at.
I was unable to run the gpu caps viewer from inside the bottle and get a meaningful console output.
Instead, I ran with:
WINEPREFIX=/home/evert/.local/share/bottles/bottles/Insta360 WINEDEBUG=+nvcuda WINEDEBUG=+loaddll wine GPU_Caps_Viewer.exe

To save you a lot of reading, this seems important:

0024:fixme:system:NtUserQueryDisplayConfig flags 0x2, paths_count 0x149d3d0, paths 0x27f6708, modes_count 0x149d3d4, modes 0x27f67a0, topology_id (nil) semi-stub
warn:  DXGI: Found monitors not associated with any adapter, using fallback
0024:trace:loaddll:build_module Loaded L"C:\\windows\\system32\\nvcuda.dll" at 646A0000: builtin
0024:trace:loaddll:load_so_dll Loaded L"\\??\\C:\\windows\\system32\\nvcuda.dll" at 00000000: builtin
0024:fixme:nvcuda:load_functions Wine cannot find the cuda library, CUDA support is disabled.
0024:trace:loaddll:free_modref Unloaded module L"C:\\windows\\system32\\nvcuda.dll" : builtin
0024:trace:loaddll:build_module Loaded L"C:\\windows\\system32\\msimg32.dll" at 7F940000: builtin
0024:trace:loaddll:free_modref Unloaded module L"C:\\Program Files (x86)\\Geeks3D\\GPU_Caps_Viewer\\gxcplugins\\plugin_gxc_gpumon_x32.dll" : native
0024:trace:loaddll:free_modref Unloaded module L"C:\\windows\\system32\\OpenCL.dll" : builtin
0024:fixme:kernelbase:AppPolicyGetProcessTerminationMethod FFFFFFFA, 0149FEB8

I'll continue poking the dragon a bit on this side, but if you have any insight on why nvcuda is not loading, it would be highly appreciated.

[evertvorster]

Here is another interesting data point.
When running the install script, everything appears fine:

[evert@Evert 0.7.19]$ WINEPREFIX=/home/evert/.local/share/bottles/bottles/Insta360 ./setup_nvlibs.sh install
[1/2] nvcuda :
    Creating DLL override... 
    Creating symlink to nvcuda.dll.so... 
[2/2] nvapi :
    Creating DLL override... 
    Creating symlink to nvapi.dll... 
[1/3] 64 bit nvcuda :
    Creating DLL override... 
    Creating symlink to nvcuda.dll.so... 
[2/3] 64 bit nvoptix :
    Creating DLL override... 
    Creating symlink to nvoptix.dll.so... 
[3/3] 64 bit nvapi64 :
    Creating DLL override... 
    Creating symlink to nvapi64.dll... 
Symlinks created in /home/evert/.local/share/bottles/bottles/Insta360. Do NOT remove this source folder!
OBS! NVML is NOT enabled by default. See Readme_nvml.txt for info or run nvml_setup.sh
You need to REMOVE old overrides if older version of nvml have been used in /home/evert/.local/share/bottles/bottles/Insta360

However, when I run the uninstall script, it appears that a symlink to nvcuda has never been made:

[evert@Evert 0.7.19]$ WINEPREFIX=/home/evert/.local/share/bottles/bottles/Insta360 ./setup_nvlibs.sh uninstall
[1/2] nvcuda :
    Removing override... 
    Removing symlink... 
[2/2] nvapi :
    Removing override... 
    Removing symlink... 
[1/3] 64 bit nvcuda :
    Removing override... 
    Override does not exist for nvcuda, trying next...
    Removing symlink... 
[2/3] 64 bit nvoptix :
    Removing override... 
    Removing symlink... 
[3/3] 64 bit nvapi64 :
    Removing override... 
    Removing symlink... 
Rebooting prefix!
wine: configuration in L"/home/evert/.local/share/bottles/bottles/Insta360" has been updated.
All done!

It seems to me that instead of just showing "Creating symlink to ..." the script should show the actual command run to ease troubleshooting. It does, however, explain why the GPU cap program is not finding the nvcuda...

[SveSop]

The "error" when running uninstall is something i could have fixed if i bothered i guess, but it is harmless.
To explain the gist of what this script does when uninstalling, is that it removes the symlinks then removes any dll overrides for wine. For nvapi, the 32 and 64-bit dll's are named different eg: nvapi.dll and nvapi64.dll, so it needs two "overrides" - one for nvapi and one for nvapi64..
For nvcuda, the 32-bit and 64-bit has the same name, so it simply removes the override to nvapi 2 times, where the 2nd time it tries it "fails" as it is not listed as an override anymore. This does not matter as it WAS removed when running the first time. You can verify this if you run winecfg and look at "Libraries" tab.

So.. Nope, it most likely works just fine

However, what does NOT work fine ref. to your snippit from the log is this:
0024:fixme:nvcuda:load_functions Wine cannot find the cuda library, CUDA support is disabled.

nvcuda loads libcuda.so.1 from your linux system as seen here:
https://github.com/SveSop/nvidia-libs/blob/master/dlls/nvcuda/nvcuda.c#L730-L736

Could you try to run: ldconfig -p|grep -i libcuda
You should have something in the lines of

        libcuda.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libcuda.so.1
        libcuda.so.1 (libc6) => /lib/i386-linux-gnu/libcuda.so.1
        libcuda.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libcuda.so
        libcuda.so (libc6) => /lib/i386-linux-gnu/libcuda.so

The placement may vary depending on distro, and it could be that some distro's or driver install issues does NOT create the libcuda.so.1 symlink perhaps? This is on Ubuntu, but as i said, some distro's may use other folders for libraries. This should not matter, as nvcuda simply loads libcuda.so.1 from where the linker (LD) sais the file is. If LD is unaware of libcuda.so.1 -> No load.

[evertvorster]

Hi there!
Thanks for the explanation, it does indeed make sense.

As for libcuda location on my system (Arch)

[evert@Evert ~]$ ldconfig -p|grep -i libcuda
        libcudart.so.12 (libc6,x86-64) => /opt/cuda/lib64/libcudart.so.12
        libcudart.so (libc6,x86-64) => /opt/cuda/lib64/libcudart.so
        libcudadebugger.so.1 (libc6,x86-64) => /usr/lib/libcudadebugger.so.1
        libcudadebugger.so (libc6,x86-64) => /usr/lib/libcudadebugger.so
        libcuda.so.1 (libc6,x86-64) => /usr/lib/libcuda.so.1
        libcuda.so (libc6,x86-64) => /usr/lib/libcuda.so

So, I'll go poking around and see why wine is not able to find cuda, it may very well be specific to my system.

Thanks again for your help in troubleshooting this issue with me.

[evertvorster]

A couple of very noticeable differences I see between Arch and Ubuntu, is that Ubuntu packages a 32bit and a 64bit version of cuda, while Arch only packages the 64bit version.

On Arch there is also a libcudart and a libcudadebugger. Could it be that the script gets confused and links to the wrong library? That is one of the things I will investigate from my end.

[evertvorster]

One more thing that could be tripping up things here.
Arch packages 32bit libraries into /usr/lib32, and 64bit libraries into /usr/lib, and there is also a symlink usr/lib64 which points to /usr/lib.

[evertvorster]

One more data point... for some reason the nvapi tests now fails, no matter what I do...

-------------------------------------------------------------------------------
Sysinfo methods succeed against local system
-------------------------------------------------------------------------------
../../../dxvk-nvapi/tests/nvapi_system.cpp:75
...............................................................................

../../../dxvk-nvapi/tests/nvapi_system.cpp:135: FAILED:
  REQUIRE( nvAPI_GPU_GetMemoryInfoEx )
with expansion:
  nullptr

===============================================================================
test cases:  1 |  0 passed | 1 failed
assertions: 44 | 43 passed | 1 failed

[evertvorster]

Sorry for the many mails.
I managed to get the nvapi tests to pass again.
Rebuilt the nvidia-libs from source.
Wiped and re-installed wine from the system.
Uninstalled and then deleted the nvidia-libs from the bottle.
Copied in, and then installed the newly built nvidia-libs into the bottle
Installed the nvidia-libs with the script, and then ran the tests, and nvapi now passes again.
I still get the error message which states that the system is unable to find nvcuda.

[evertvorster]

This may be a silly question, but I see that there are two parts to what is being built by the source.
When the sources are built, there is a /lib, and a /lib64 directory, inside of each there is a wine subdirectory, under which there is a unix and a windows subdirectory.

The install program links to the windows directory, but what is supposed to happen to the contents of the unix sub-subdirectory in lib and lib64?

[SveSop]

I know this can be somewhat convoluted to understand.
The package nvidia-libs contains both "windows" dll's and "unix" dll's. I wont go into detail on how this works, but a quick summary is that nvapi is a "pure PE/windows dll" and thus will symlink to the "windows" folder, and nvcuda is a "ELF/Linux so.dll" and thus will symlink to the "unix" folder.

To use 32-bit windows programs like gpu caps viewer, 32-bit nvidia driver libraries must be installed.
You do not seem to have the 32-bit nvidia driver libraries installed when i look at the output from ldconfig -p| grep -i libcuda you posted above. How to install the 32-bit nvidia libraries on archlinux i do not know.

[evertvorster]

Hi there...
We seem to be getting there, step by step.
I installed the lib32-nvidia-utils package in Arch, which provides a 32bit version of cuda.
Indeed, when running gpu caps viewer in bottles, CUDA is indeed available now:

Unfortunately, it does not seem to detect the CUDA card with GPU-Z, or the application that I want to run with CUDA support, namely Insta360 studio.
There are run in the same bottle where the GPU-caps viewer was run. I suspect that there is still an issue with the 64bit version of CUDA on this machine.

Also, if I run GPU Caps outside of the bottle, it also does not detect CUDA, but the error is different than before:
Console_Output_2024-09-29.txt

[SveSop]

I could not really get GPU-Z to work either... Bare in mind that these "info" programs usually requires a bit more barebones system functions that ANY of the nvapi/nvcuda++ libraries provides.
I do not think i will spend overly amount of time getting GPU-Z to work, as this may require a bit more than just some libraries in the wrong place.

It seems as insta360 requires that you purchased a camera to download the "free" software.. and i am not about to do that. Perhaps you could provide some logs from that instead of GPU-Z (that most likely wont work too well)?

[evertvorster]

That makes sense.

I will be happy to provide you a copy of the installer. Send me a mail at evorster@gmail.com, and I will share it via a WeTransfer link.

However, I think I have been looking in the wrong direction. I see on the application window that CUDA is enabled, but hardware encoding and decoding are still disabled. This may be the application wanting to talk to the nvidia drivers through a different channel that is not provided by CUDA.

Since we managed to get the card talking to GPU Caps, I guess we can close this report, and I will start looking elsewhere to see if I can make this application do hardware encoding and decoding through wine.

[SveSop]

Hardware encoding/decoding uses nvcuvid and nvencodeapi and is not provided. There was some implementation of this in wine-staging at a time, but it was not really maintained and when i took over my own branch of nvcuda, i had no huge interest in doing this.

So, it could be quite likely that insta360 that seems to be some sort of video encoding software attempts to use one of these libraries, and that is the reason it does not work.

[evertvorster]

Hi there. You are quite right. In my preliminary searches on this topic, your name actually pops up quite a bit, so I am going to take your word for it that hardware encode and decode is simply not supported through wine. Video encode and decode with hardware is completely off-topic to this bug report, too.

I did get CUDA support on the video card, and for that I do thank you.
I'll go ahead and close this bug report, as that issue is solved.

[SveSop]

As a sidenote, you can try the simplified nvcuda archive here: https://github.com/SveSop/nvcuda/releases/download/v0.1/nvcuda-0.1.tar.gz

There is no need to use the whole nvidia-libs package for cuda, as bottles already have dxvk and dxvk-nvapi as installable components. Simply copy the x32/nvcuda.dll into the bottles prefix in windows/syswow64 and the x64/nvcuda.dll into windows/system32

This should make CUDA work in a bottle without the need for using nvidia-libs at all.