App runs faster in Nsight Systems than standalone (Vulkan)

My app runs 50fps faster when profiling in Nsight Systems than standalone.

Under Collect Vulkan trace:
Note: Nsight Systems suppresses loading of explicit Vulkan layers

Is that the difference?

Which flags are set/cleared to suppress those layers?

1 Like

On Windows, Nsight Systems actively disables some known implicit and explicit layers from other NVIDIA and 3rd party tools.

On Linux, Nsight Systems forces the values of VK_LAYER_PATH and VK_INSTANCE_LAYERS environment variables, which may interfere with the finding and/or loading of other layers.

On both platforms, it does not change the parameters passed to vkCreateInstance and vkCreateDevice, so if you are using those to load layers, they will need to be disabled manually. Note that the change to VK_LAYER_PATH might make some layers unfindable by the the Vulkan loader, effectively disabling them.

As for the performance “increase”, if it is not related to Vulkan layers being disabled, it could also be a previously fixed bug that would force the Windows driver module to accidentally give higher power and computation priority to the application being profiled (causing incorrect results). It is fixed in the latest version of Nsight Systems, so if you have an older version, try updating and seeing if the issue goes away.

Thank you!

I am on Windows 10 with Nsight Systems, Nsight for Visual Studio, and Nsight Graphics installed, plus some other tools.

Nsight Systems 2020.1.1.65-085319d Windows-x64
Nsight for Visual Studio 2019.4.1.19317
Nsight Graphics 2020.1.0.0 (Build 27904074)
NVidia driver 442.19
NVidia GeForce RTX 2060
VTune 2020 (Build 605129)
RenderDoc 1.6
PIX 9.29.1962.0000 (June 2010) (not using this)

To vkCreateInstance, I pass only VK_KHR_SURFACE_EXTENSION_NAME and VK_KHR_WIN32_SURFACE_EXTENSION_NAME.
To vkCreateDevice, I pass only VK_KHR_SWAPCHAIN_EXTENSION_NAME.

In my registry, I see all these keys disabled (zero):
HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ExplicitLayers
HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ImplicitLayers

No Vulkan environment variables set other than VK_SDK_PATH and VULKAN_SDK

How can I disable the other layers that Nsight Systems disables? (in Windows)

Of the list you gave, Nsight Systems disables the layers for:
Nsight VSE
Nsight Graphics
RenderDoc

To disable Explicit Layers, you need to just not include them in the calls to vkCreateInstance / vkCreateDevice - which seems to be the case for you, as you’re only calling extensions and not layers.

To disable Implicit Layers, the process is slightly more complicated:

  1. Go to HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\Vulkan\ImplicitLayers
  2. Each of the values under this key is a file path to a layer’s json manifest
  3. Follow the file paths to the json file whose layer your want to disable
  4. Here, there are three options:
    4a. The layer manifest contains a value named “disable_environment”
    5a. To disable this layer, simply set the environment variable named to the value named - for example, to disable the Nsight VSE layer, set DISABLE_VK_LAYER_NV_nsight=1 in the environment
    4b. The layer manifest contains a value named “enable_environment”, but does not contain “disable_environment”
    5b. To disable this layer, just make sure that the variable named is not set in the environment
    4c. Neither value exists in the manifest
    5c. This should not be allowed, but some implicit layers choose to do this - the only way to disable them in this case is to either remove the registry value for them; or to edit the layer manifest to add a “disable environment” value
  5. Note that if you remove or edit these files, the proper function of the tool whose layer you removed or changed is not guaranteed, and Vulkan support in those tools might be compromised

Hope this helps!

Thank you!

How does Nsight Systems disable these at runtime? The .json files haven’t changed on disk since I installed those tools.

Is there a fourth option where both enable_environment and disable_environment are set?

This is what I have:

C:\Program Files (x86)\NVIDIA Corporation\Nsight Visual Studio Edition 2019.4\Monitor\Common\Injection64\VK_LAYER_NV_nsight.json

"enable_environment": {
      "ENABLE_VK_LAYER_NV_nsight": "1"
    },
    "disable_environment": {
      "DISABLE_VK_LAYER_NV_nsight": "1"
    }

C:\Program Files\RenderDoc\renderdoc.json

"enable_environment": {
      "ENABLE_VULKAN_RENDERDOC_CAPTURE": "1"
    },
    "disable_environment": {
      "DISABLE_VULKAN_RENDERDOC_CAPTURE_1_6": "1"
    }

For RenderDoc, I need to disable two?
DISABLE_VULKAN_RENDERDOC_CAPTURE
DISABLE_VULKAN_RENDERDOC_CAPTURE_1_6

I have different performance results running standalone, launched in Visual Studio, and launched in Nsight Systems.

Does each app know of the layers of other apps to disable? It could find them through the registry or globally disable all implicit layers and reenable the important ones?