Problem running OptiX 7.6 in WSL

Gaming and Visualization Technologies Visualization OptiX
1

Hi,

I’m having a problem executing any OptiX 7.6 code in WSL, Ubuntu 22.04 LTS. Any examples, from the SDK, optix7course, or the OptiX_Apps, all result in error code 7805, OPTIX_ERROR_ENTRY_SYMBOL_NOT_FOUND.

The only hint I have is that, by modifying the intro_driver example in Optix_Apps, I get an error message from dlerror() after attempting dlsym() to get optixQueryFunctionTable. This error is “/usr/lib/wsl/drivers/nvdmig.inf_amd64_f1f00df48246f9a3/libnvoptix.so.1: cannot open shared object file: No such file or directory”

It’s true that the libnvoptix.so.1 isn’t in this directory, and is in /usr/lib/wsl/lib/. Changing the string argument to dlsym() does result in the expected “no such symbol” kind of error, so it is getting the file.

This is on a completely clean WSL 2 Ubuntu, with Windows drivers fully updated, with CUDA and OptiX 7.6 installed from the WSL Ubuntu versions (CUDA via runtime file instead of deb, since deb didn’t include NVCC).

Anyone have any ideas on what’s going on?

Thanks!
J

nvidia-smi output

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 527.92.01    Driver Version: 528.02       CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0  On |                  N/A |
| N/A   45C    P8    18W / 139W |   1188MiB /  8192MiB |      4%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
2

Hi @james.burkert, welcome!

Apologies, but OptiX is not yet included in the WSL driver component, so it is not expected to work. Do you have a native Windows or Linux environment you can continue development in? We hope this can be done soon, as I know there are others interested and waiting.

–
David.

3

Aww shucks, somehow I didn’t get that from the docs. Yep, I’ll just use a different dev environment.

Thanks,
J

4

I’ve encountered the same issue. My employer wants us to user company machines (Windows), but I can’t get libnvoptix.so in WSL yet, not even in the development docker container I use. If the driver libnvoptix.so libraries are missing from the host (WSL) then they don’t get mapped into the container.

5

Hi @adzngjeagf, our recommendation for the time being is to make sure the native (non-WSL) Windows drivers are installed, and rely on those to build a non-WSL specific app. We don’t have a lot of experience with OptiX in Docker containers, but I’ve heard multiple reports of people having success with the native drivers on Docker.

–
David.

1 Like
6

Sorry I’ve been cut off from this account due to the 2FA setup with an inaccessible email address. Thanks for the reply.

I’m just not accustomed to using Windows for HPC things like this. My current setup is in a development container that I just run on Docker CE for Ubuntu 22.04 using the CUDA+CUDNN 12.0 image from DockerHub. I have to add the environment variable -e NVIDIA_DRIVER_CAPABILITIES=all and everything works.

I map in the Xauth and can do all the display from inside too. Now that the new WSL 22.04 has Xwayland built-in it would be pretty fabulous to get support in Docker for Windows too. Fingers crossed.

7

Any update to Optix support in WSL2?

8

Welcome @chris.schwindt,

I believe we’re not yet packaging OptiX into the WSL2 driver. I believe this is hung up on a redesign of the driver packaging and delivery process, which is why it’s taking such a long time.

I have heard rumors that people have been able to get OptiX to work in WSL2 via manual install. This is unofficial and subject to change, so your mileage may vary, but here are some steps that may work for you:

Running OptiX Applications on WSL 2

Install WSL 2 and enable CUDA

Follow the canonical methods for installing WSL, display driver, and CUDA Toolkit within WSL

  • As mentioned in the docs, do not install a Linux Display driver in WSL, this will break the mapping of libcuda.
    There are CUDA Toolkit downloads specifically for WSL that will not attempt to install a driver, only the toolkit.
    You can also deselect the driver in a normal version of the toolkit.

Obtain OptiX / RTCore libraries for Linux

  • Download and extract libraries from the linux display driver.
  • You can run the driver installer in WSL using ./[driver filename].run -x which will unpack the driver but not install it.
  • Copy libnvoptix.so.XXX.00, libnvidia-rtcore.so.XXX.00, and libnvidia-ptxjitcompiler.so.XXX.00 into C:/Windows/System32/lxss/lib where XXX is the driver version.
  • Rename libnvoptix.so.XX.00 to libnvoptix.so.1
  • Rename libnvidia-ptxjitcompiler.so.XXX.00 to libnvidia-ptxjitcompiler.so.1
  • Do not rename libnvidia-rtcore.so.XXX.00
  • Be aware that future drivers may need additional libraries that will need to be copied.

Building an OptiX Application

  • You may need to add /usr/local/cuda/bin to your PATH to access NVCC, but do NOT add /usr/local/cuda/lib64 to LD_LIBRARY_PATH as you normally would when installing the CUDA toolkit. libcuda and other libraries are passed through from C:/Windows/System32/lxss/lib where you placed the OptiX and RTCore libs.
  • Instead, add /usr/lib/wsl/lib to your LD_LIBRARY_PATH to pick up CUDA, OptiX, etc.

Running an OptiX Application

  • With LD_LIBRARY_PATH set per the previous step, you should be able to run an OptiX executable.
  • You may need to rebuild the WSL cache. You can do so by quitting any WSL sessions and running wsl --shutdown from Powershell, then starting a new WSL session. Failing to reset the cache may lead to strange load paths.
  • You may verify paths are correct using strace, e.g., strace -o trace ./bin/optixHello

–
David.

1 Like
9

Aha! This looks very promising! (a few minutes later) Thank you! It actually worked. However, one tweak for those who are hamstrung by their companies cybersecurity policies and don’t allow copying things into C:/Windows/System32/lxss/lib. You can also copy the files and do the renaming in /usr/lib/wsl/lib, since in there you can use sudo.

So, David…THANK YOU!

3 Likes
10

Yes! Glad it worked and thanks for the followup tip. Credit for these instructions really goes to my teammate Daniel for doing all the legwork and testing and writing them up recently, I just passed them along. So, thank you, Daniel! ;)

–
David.

11

Yes, THANK YOU, Daniel! Note that when you shutdown and restart the WSL VM, with my method of using /usr/lib/wsl/lib, you will have to redo it each time as starting the VM up again seems to restore the /usr/lib/wsl/lib to its original pristine state. So I just made a script to quickly do it each time I need to.

12

Hi David
This fix was indeed very helppful and promising, but I am running into the following error when trying to run optixHello:

OptiX Error: ‘Unknown error (Details: Function “RTresult _rtContextLaunch2D(RTcontext, unsigned int, RTsize, RTsize)” caught exception: Encountered a rtcore error: m_api.compileModule( context, options, inputSerializedModuleBuffer, bufferSize, compiledModule ) returned (7): Compile error)
(/home/desnt/OPTIX/SDK/optixHello/optixHello.cpp:108)’
Segmentation fault

I am trying to run SDK for OptiX 6.5 specifically since it is the latest version that compiles VMD. I am not good with strace or any debugging really, so I cannot get anywhere with this issue. When using Tachyon-OptiX in VMD I get exactly the same error.

I am running:

  • KERNEL: Linux DESKTOP-DD89L6Q 6.1.21.2-microsoft-standard-WSL2 #1 SMP PREEMPT_DYNAMIC (6.1 WSL2 kernel with minimal modifications (Ryzen power-states fix for freezes))
  • cuda-12.3 patch 2
  • OptiX 6.5
  • 551.61 Windows driver
  • 550.54 Linux driver for libnvoptix (do the version need to match exactly? can they match exactly?)
  • Lenovo laptop with RTX 3060 (Integrated graphics disabled in BIOS)

Would greatly appreciate any help. Thanks in advance.

13

Hi @desgraciadnot,

OptiX 6.5 has different DLL/so filenames than OptiX 7+, so the above instructions are probably not exactly correct. The .so file that is packaged with OptiX 6.5 is named something like optix.so.6.5.0 on Linux or optix.6.5.0.dll on Windows.

Also note that OptiX 6.5 is not officially supported on CUDA 12, nor CUDA 11. Our recommendation is to use CUDA 10 with OptiX 6.5 - this may be the cause of your error and not a missing library, as Detlef mentioned in this thread: Problem with running OptiX 6.5 program. "invalid value for --gpu-architecture"

If you are missing a library, now’s a good time to start playing with strace and learn how to use it. It’s a firehose of information, but it’s not hard at all. It shows you what files your app tries to load, and whether or not it succeeds. There are command line options for filtering out system events that you can read about in the man pages, but if you just filter/search for ‘optix’ you will probably see which library it’s looking for and failing to find.

Another good alternative to strace on Windows is to use the Sysinternals Suite’s procmon utility - it will also show you every system event including every file open request. Procmon might be easier to explore than strace if you’re not used to command line tools, but procmon does take a little getting used to in order to show a single process and filter for file access.

–
David.

1 Like
14

Thank you!
I have tried again with CUDA 10 and GCC 8. I got an error “Failed to load OptiX library”. Strace showed that optixHello looked in a bunch of places for libnvoptix.so.1, found it in /usr/lib/wsl/lib and then tried to get libnvoptix.so.1 from the DriverStore. CUDA 12 did not do this (it went to get libcuda.so.1 from DriverStore).
After doing naughty things (tampering with drivers), I put libnvoptix.so.1 into the DriverStore… and was back to the same error at RTresult _rtContextLaunch2D.
At that point the trace is a bunch of futexes and an error:

futex(0x559b9ca22f78, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x559b9ca22f80, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x559b9cbead28, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x559b9cbead28, FUTEX_WAIT_PRIVATE, 2, NULL) = 0
futex(0x559b9cbead28, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x559ba06699c0, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, FUTEX_BITSET_MATCH_ANY) = 0
futex(0x559ba06699c8, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x7fe5267d21f0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(2, “OptiX Error: '”, 14) = 14
write(2, “Unknown error (Details: Function”…, 329) = 329

Again, at my current level this tells exactly nothing to me. All the libs seem to be loaded correctly as far as I can tell at least (except libnvoptix-sdk).
This is very cool learning experience, but I am not sure that getting images from VMD a couple of seconds faster is really worth it…
Appreciate your help though. Thanks.

15

Hi,
so I tried it as well but I am not quite successfull and have no clue why. The application will initialize optix correctly as far as I can tell but then will fail when trying to compile the optix modules:
[ 4][ KNOBS]: All knobs on default.

[ 4][ DISKCACHE]: Opened database: “/var/tmp/OptixCache_nikelsm/optix7cache.db”
[ 4][ DISKCACHE]: Cache data size: “0 Bytes”
[ 4][ DISKCACHE]: Cache miss for key: ptx-6452-key569e3f6b45859a57913dfffc1c54fb55-sm_89-rtc1-drv551.61

[ 4][ COMPILER]: Info: Pipeline parameter “params” size is 16 bytes

[ 2][ COMPILER]: COMPILE ERROR: Module compilation failed
Info: Module Statistics
payload values : 0
attribute values : 0
Pipeline configuration: (default)
Info: Properties for entry function “__raygen__draw_solid_color”
semantic type : RAYGEN
trace call(s) : 0
continuation callable call(s): 0
direct callable call(s) : 0
basic block(s) : 1
instruction(s) : 75
Info: Compiled Module Summary
non-entry function(s): 0
basic block(s) : 0
instruction(s) : 0

Caught exception: OPTIX_ERROR_INTERNAL_COMPILER_ERROR: Optix call ‘optixModuleCreate( context, &module_compile_options, &pipeline_compile_options, input, inputSize, LOG, &LOG_SIZE, &module )’ failed: /home/nikelsm/NVIDIA-OptiX-SDK-8.0.0-linux64-x86_64/SDK/optixHello/optixHello.cpp:150)
Log:
COMPILE ERROR: Module compilation failed
Info: Module Statistics
payload values : 0
attribute values : 0
Pipeline configuration: (default)
Info: Properties for entry function “__raygen__draw_solid_color”
semantic type : RAYGEN
trace call(s) : 0
continuation callable call(s): 0
direct callable call(s) : 0
basic block(s) : 1
instruction(s) : 75
Info: Compiled Module Summary
non-entry function(s): 0
basic block(s) : 0
instruction(s) : 0
Info: Pipeline parameter “params” size is 16 bytes

The trace is also not helpful:
rite(2, “[”, 1) = 1
write(2, " 4", 2) = 2
write(2, “][”, 2) = 2
write(2, " ", 1) = 1
write(2, " ", 1) = 1
write(2, " ", 1) = 1
write(2, " ", 1) = 1
write(2, “COMPILER”, 8) = 8
write(2, "]: “, 3) = 3
write(2, “Info: Pipeline parameter "params”…, 51) = 51
write(2, “\n”, 1) = 1
clock_gettime(CLOCK_MONOTONIC, {tv_sec=2933, tv_nsec=591882835}) = 0
clock_gettime(CLOCK_MONOTONIC, {tv_sec=2933, tv_nsec=591951995}) = 0
futex(0x7fcb262cf0ac, FUTEX_WAKE_PRIVATE, 2147483647) = 0
lseek(2, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
write(2, “[”, 1) = 1
write(2, " 2”, 2) = 2
write(2, “][”, 2) = 2
write(2, " ", 1) = 1
write(2, " ", 1) = 1
write(2, " ", 1) = 1
write(2, " ", 1) = 1
write(2, “COMPILER”, 8) = 8
write(2, "]: ", 3) = 3
write(2, “COMPILE ERROR: Module compilatio”…, 688) = 688
write(2, “\n”, 1) = 1
clock_gettime(CLOCK_MONOTONIC, {tv_sec=2933, tv_nsec=599486714}) = 0
futex(0x7fcb40e33210, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(2, "Caught exception: ", 18) = 18
write(2, “OPTIX_ERROR_INTERNAL_COMPILER_ER”…, 1014) = 1014
write(2, “\n”, 1) = 1
getpid() = 32860
ioctl(3, _IOC(_IOC_READ|_IOC_WRITE, 0x47, 0xd, 0x20), 0x5595d2772ec0) = 0
ioctl(3, _IOC(_IOC_READ|_IOC_WRITE, 0x47, 0xd, 0x20), 0x5595d2772ec0) = 0
ioctl(3, _IOC(_IOC_READ|_IOC_WRITE, 0x47, 0xd, 0x20), 0x5595d2772ec0) = 0
ioctl(3, _IOC(_IOC_READ|_IOC_WRITE, 0x47, 0xd, 0x20), 0x5595d2772ec0) = 0
munmap(0x7fcb21800000, 78656520) = 0
close(3) = 0
lseek(9, -328, SEEK_CUR) = 16375480
exit_group(1) = ?
+++ exited with 1 +++

From the error this looks like an internal compiler error but I guess there is just something messed up with the install.
Does anyone have an idea?

16

Ok, looks like turning the system off and on again fixed it :)

1 Like
17

With the steps provided I was able to run our OptiX application on WSL 2, however creation of the OptiX denoiser failed:

optixDenoiserCreate returns OPTIX_ERROR_INTERNAL_ERROR and outputs
DENOISER: Unable to load denoiser weights

I found it works after copying nvoptix.bin from the linux driver library files to C:/Windows/System32/lxss/lib.

1 Like