NSIGHT: Not recording cudaMem API calls

Developer Tools Nsight Visual Studio Edition
1

GPUs: Two GeForce RTX 2080s

  • Issue occurs with SLI enable and disabled.
  • Latest GeForce Drivers
  • CUDA10
  • NSIGHT 6.0, 5.6, 5.3

Setup a Performance Analysis

  • Issues occurs both Local and Remote.
  • Trace Process Tree
  • Trace Settings - CUDA, 4/4 (no other traces selected).

Steps:

  • Start Application control (Via NSIGHT).
  • Run application.
  • Exit the application.
  • Stop the capture.
  • stop the application control.
  • Report window pops up.
    • In the drop-down there is no item for CUDA Memory Copies, or CUDA Memory sets.
    • Under the Runtime API Call Summary there are no cudaMemset or cudaMalloc, etc.

Working Scenario:

  • Same system as above except:
    • Two Quadro P2000s
    • Latest Quadro Driver.

Is there some setting or a different driver that I need for CUDA calls to be captured by NSIGHT?

Bug_P2000_Working_002.png
Bug_P2000_Working_002.png1157×468 49.2 KB

Bug_Rtx2080_NotWorking_001.png
Bug_Rtx2080_NotWorking_001.png981×711 49.3 KB

Bug_Rtx2080_NotWorking_002.png
Bug_Rtx2080_NotWorking_002.png760×477 27.3 KB

Bug_P2000_Working_001.png
Bug_P2000_Working_001.png970×707 52.8 KB

2

FYI: Bug 2498974 - https://developer.nvidia.com/nvidia_bug/2498974

3

NsightVSE Analysis Trace

  • Supports Kepler, Maxwell, and Pascal GPUs.
  • For Pascal and later GPUs (including RTX / Turing GPUs), see Nsight Systems or Visual Profiler 

NsightVSE Analysis Profiling on Turing GPUs

  • is available for Volta and later GPUs (including RTX / Turing GPUs) when using the Nsight Compute profiler. NOTE that Nsight Compute is also available as a stand-alone and CLI profiler

Nsight Systems : https://developer.nvidia.com/nsight-systems
Nsight Compute : https://developer.nvidia.com/nsight-compute
Visual Profiler: https://developer.nvidia.com/nvidia-visual-profiler

4

Sorry for my lack of understanding - Are you saying NSIGHT VSE doesn’t support the Geforce RTX 2080?

The page for NSIGHT VSE (https://developer.nvidia.com/nsight-visual-studio-edition-requirements) identifies the 20-series GeForce GPUs as supported.

Or are you saying there’s a problem or only partial support and to try these other tools for now?

What’s the difference?

  • If not supported, I wont bother installing NSIGHT VSE and just use the other tools.
  • If temporarily not supported / a bug, then I’ll keep both in hopes that I wont need the second tool later on.
5

It’s just the Analysis Trace and (Legacy) CUDA Profiler that do not support RTX GPUs.
The NsightVSE Nsight Compute profiler does support RTX GPUs.
The Next-Gen Debugger, Graphics Debugger, and MemChecker also support RTX GPUs

https://developer.nvidia.com/nsight-visual-studio-edition
mentions

and
https://developer.nvidia.com/nsight-visual-studio-edition-features
mentions

Nsight Compute will be the profiler for late model GPUs (supportting Pascal and later GPUs, both integrated in NsightVSE for windows and standalone for all supported OSs)

Nsight Systems will be the trace tool for late model GPUs (supporting Pascal and later GPUs on all supported OSs)

6

Thank you for the clarification.

  • I hope there are plans to consolidate ;-)
  • The tools are free to begin with, so a big thank you for that!!

Cheers