Problems launching or attaching a Vulkan application in Linux

Hello.

I’m trying to debug my Vulkan application with Nsight Graphics.
The way I normally run it is by launching a simple script that first sets the environment variables needed for the application to work properly and then proceed to launch it.

The run script is the following:

export VULKAN_SDK=./extern/Vulkan
export PATH=$VULKAN_SDK/bin:$PATH
export LD_LIBRARY_PATH=$VULKAN_SDK/lib:$LD_LIBRARY_PATH
export VK_LAYER_PATH=$VULKAN_SDK/etc/vulkan/explicit_layer.d
./build/app

I’ve tried two strategies that both failed.

  1. Directly launching the app with the environment variables set.
    Application Executable: /home/user/Documents/vulkan/build/app
    Working Directory: /home/user/Documents/vulkan
    Environment: PATH=/home/user/Documents/vulkan/extern/Vulkan/bin; LD_LIBRARY_PATH=/home/user/Documents/vulkan/extern/Vulkan/lib; VK_LAYER_PATH=/home/user/Documents/vulkan/extern/Vulkan/etc/vulkan/explicit_layer.d
    Automatically Connect: On

I even wrote everything in absolute paths and without referring to other environment variables using ‘'. Nothing worked (Obviously I first tried with relative paths and referencing other environment variables with '’).
The result is “Failed to connect (host: localhost, pid: 7376). The target may already have exited.”
EDIT: I’ve disabled the validation layers and it no longer gives me this error right away, but it stays indefinitely “Trying to connect to process on host”.

  1. Launch the run script without connecting automatically and try to attach the process later.
    Application Executable: /home/user/Documents/vulkan/run
    Working Directory: /home/user/Documents/vulkan
    Automatically Connect: Off

However, the process list appears empty.

Details:
OS: Ubuntu 18.04
Nsight Graphics version: 2019.2.1.0
Nvidia Drivers: 410.104
Graphics Card: Nvidia GTX 950M

What is the correct Nsight Graphics setup to debug my application?

Hello,

Thank you for your feedback on Nsight Graphics. From looking at your steps in the report you sent, it appears that the setup you are using to debug your application looks fine. Your description seems to be the way to set the environment variable when trying to launch an application for debugging.

Would it be possible for you as a next step to use our latest Nsight Graphics 2019.5 release and try your steps again? Our 2019.2 release is a few months older and 2019.5 has a number of fixes that may have resolved your issue.

You can find 2019.5 by going to the following location https://developer.nvidia.com/nsight-graphics. Please let us know how it goes with 2019.5.

Regards,

Darrell

Hi Darrel,

Since then I’ve updated Nsight Graphics to 2019.5. I’ve also updated the driver to 430.50.
The problem remains.
If I launch Nsight through a terminal with the environment variables set, and then setup the application executable and working directory (with automatically connect ON), my app is launched and works correctly but it still stays indefinitely “Trying to connect to process on host”.

Regards,
Daniel

Hello,

Thanks for the update. Our engineer tried launching a Vulkan sample on their Linux machine with the same environment settings as yours, however didn’t see the same issues. Is it possible for you to share your app with us for further investigation?

Regards,

Darrell

Hello,
Thanks for investigating. I’ve discovered that it’s not a problem with environment variables. I can say this because I’ve tried running a known Vulkan application which requires none and it still didn’t work, displaying the “Trying to connect to process on host” message forever. You can get this Vulkan app here: https://github.com/SaschaWillems/Vulkan-glTF-PBR.
I’ve also tried an OpenGL application and it worked perfectly, so this is definitely a Vulkan issue.
Cheers,
Daniel

I would suggest your engineer to use a clean Ubuntu 18.04 image with the 430.50 driver, (for convenience a Virtual Machine might be better) because that would rule out or evidentiate that your software is relying on some packages that may not be installed or set up as expected.

I must add that I’ve tested the same application on a competitor software (not going to mention its name, but it’s “the open-source one”) and it worked perfectly, albeit it’s not as pleasant to work with.

Let’s hope that, in the future, NVIDIA adopts open-sourcing more seriously and this type of problems may go away as the community could help itself.

Hello,

Thanks for the additional details. The developer suspects that Nomad’s interception layer wasn’t loaded somehow on your machine. The reason could be the layer manifest not being correctly installed.

Can you check the file’s existence at the following location? ~/.local/share/vulkan/implicit_layer.d/VK_LAYER_NV_nomad_release_public_2019_6_0.json

It should be deployed during the install, but might be unexpectedly removed. If this file doesn’t exist, you could try to run the script to re-install it: /host/linux-desktop-nomad-x64/VK_LAYER_NV_nomad.sh.

Let us know if this helps.

Regards,

Darrell

At the directory ~/.local/share/vulkan/implicit_layer.d/ there was only one file named VK_LAYER_NV_nomad.json

After running the script you recommended I noticed that it was trying to copy VK_LAYER_NV_nomad_release_public_2019_5_1.json (from the installed path) to ~/.local/share/vulkan/implicit_layer.d

However, cp and chmod were failling with “permission denied”, and therefore I ran it with sudo.
After that I tried running my vulkan application on Nsight and it finally worked!

Thanks for all your help.

As a side note I would like to reference that the installation guide for Linux does not mention the need for sudo anywhere (I would suppose that this copy script is being called by the “run” script mentioned in the guide). I really think you should write a note about it there. https://docs.nvidia.com/nsight-graphics/InstallationGuide/index.html#linux_installation_and_setup

By the way, 2019.6.0 is not out yet! eheh ;)

Hello,

Glad to hear that you were able to get past this issue. Our apologies for the lack of information in the installation guide about the need for sudo. We now have a bug created on our side to improve the area of checking the Vulkan layer manifest during launching.

Regards,

Darrell

I am having the same problem on a Windows machine. Is there a script on a Windows machine that I can run so that the Nomad layer is found?!?!

I have upgraded Nsight Graphics to the 2020 version and also tinkered with the Vulkan Configurator to get the following error resolved:

Failed to open VK_LAYER_NV_nomad_release_public_2019_6_1.json

Any help would be appreciated!

-Cuda Education
cudaeducation.com

Hello,

I spoke to our engineer and he recommended the following:

Have him look in HKLM\SOFTWARE\Khronos\Vulkan\ImplicitLayers. The directory there should point to his current install of Nsight and the JSON file in that directory Given that he has installed 2020 and is seeing the 2019_6_1.json error message, he may need to remove that one.

Please let me know if you have additional feedback.

Hi,

I deleted the 2019.json entry in the registry, but I still get the same error.

Please watch this new video for more detailed information: https://youtu.be/xB966qfld7I

The 2020.json files that you see in the registry are all there. I just don’t understand where it got the directory for 2019.json from! I can’t even get the 2019.json file anywhere on the internet!

I have also upgraded my Vulkan SDK version to the latest, made sure my NVIDIA drivers are up to date etc.

I personally don’t think the json file issue is causing the program to not attach to Nsight Graphics 2020.

I can share the source code with you so you can attempt it on a Windows machine at your place.

Thanks for your help!

-Cuda Education
cudaeducation.com

Hello,

We’re wondering if you are setting the working directory correctly. This is covered in: https://docs.nvidia.com/nsight-graphics/UserGuide/index.html#troubleshooting_problem___the_application_fails_to_launch.

Please have a look at this guide and let us know if this helps to get the issue resolved.

Hi guys,

I figured it out. There was a problem with my Vulkan program where it would exit prematurely if you tried to run the .exe file outside of the Visual Studio Build -> Run process. I fixed the problem of running the .exe file directly so that it wouldn’t exit prematurely and now it works fine.

Thanks for your efforts!

-Cuda Education
cudaeducation.com