Jetson TX2 (8GB) Dev Board - JetPack 4.6.2 Does not Install NSIGHT Compute

[nickjeliopoulosdev]

Hello all,

I am trying to profile some PyTorch programs on my Jetson TX2 (8GB) Devkit w/ Jetpack 4.6.2.

JetPack 4.6.2 Missing NSIGHT Compute
I have seen other users mention that the NVIDIA NSight Compute executable should be located somewhere in /opt/nvidia/, but that is not the case for me.

I used the SDK Manager to reinstall the JetPack 4.6.2 development kit components on my Ubuntu 18.04 machine, but I still do not see the NSight Compute executable in /opt/nvidia/

Remote Launch + Attach
I have also attempted to remotely launch + attach with NSight Compute 2019.5.3 on my Ubuntu 18.04 host machine, but I encounter a warning that indicates I do not have the correct permission to remotely profile.

To fix this issue, I copy pasted the CLI code from NSight Compute and ran it directly on the Jetson TX2 with sudo. This also results in a problem, because I receive an error that my GPU is unsupported.

Request
It would be great to understand why NSight Compute is not installed on my TX2, and how to fix that.
I have also attached the output of the deviceQuery sample just in case it is useful
deviceQuery.txt (2.4 KB)

Thank you for your time,
Nick

[AastaLLL]

Hi,

Based on the document below, Nsight compute cannot use Jetson as a host:

Release Notes :: Nsight Compute Documentation

But you should be able to get it on an x86 Ubuntu if you install the host component from JetPack 4.6.2.
The location is /opt/nvidia/nsight-compute/.

Thanks.

[nickjeliopoulosdev]

This post suggest I use NSight Compute 2019.5.3 Can not find Nsight Compute for JetPack 4.6 - #3 by AastaLLL

I have NSight Compute 2019.5.3 on my host system, however I encounter two problems when trying to remotely launch + attach on the TX2:

(1) NSight Compute returns error code 103, and notes that I do not have permission to profile on the remote device
(2) To resolve this, I launched the CLI command supplied by NSight Compute directly on the Jetson TX2 prefixed with sudo. This results in an error that says the TX2 GPU is unsupported.

Do you have any suggestions on how to resolve this? Let me know any further information I can supply to assist you.

[AastaLLL]

Hi,

Please make sure the CUDA toolkit between the x86 host and Jetson is identical.

Instead of using cli tool, you can try to setup the root password and remote profile it with root account:

No timeline for profiler Xavier: nvvp and nsight compute is not working in Jetson Xavier

Hi, Have you setup the root password first? The process should like this: 1.Setup root password $ sudo passwd root 2.Edit /etc/ssh/sshd_config diff --git a/sshd_config b/sshd_config index a32dc1d..d21cacd 100644 --- a/sshd_config +++ b/sshd_config @@ -30,6 +30,7 @@ #LoginGraceTime 2m #PermitRootLogin prohibit-password +PermitRootLogin yes #StrictModes yes #MaxAuthTries 6 #MaxSessions 10 @@ -54,7 +55,7 @@ # To disable tunneled clear text passwords, change to no here! #PasswordAut…

Thanks.

[nickjeliopoulosdev]

Hi there,

I took your advice and was able to initiate remote profiling with the root account, thank you for that.

There are multiple CUDA versions on my host machine. I do have CUDA 10.2, however. I am not sure if this will interfere at all:

$ ls -al /usr/local/
total 52
drwxr-xr-x 13 root root 4096 Oct 21 19:17 .
drwxr-xr-x 13 root root 4096 Sep 23 21:18 ..
drwxr-xr-x  2 root root 4096 Oct 21 14:47 bin
lrwxrwxrwx  1 root root   22 Sep 23 21:17 cuda -> /etc/alternatives/cuda
lrwxrwxrwx  1 root root   25 Sep 23 21:17 cuda-10 -> /etc/alternatives/cuda-10
drwxr-xr-x 17 root root 4096 Sep 23 21:17 cuda-10.2
lrwxrwxrwx  1 root root   25 Oct 21 19:17 cuda-11 -> /etc/alternatives/cuda-11
drwxr-xr-x 14 root root 4096 Oct 21 19:17 cuda-11.4
drwxr-xr-x  5 root root 4096 Oct 21 19:14 cuda-11.7
drwxr-xr-x  2 root root 4096 Sep 15  2021 etc
drwxr-xr-x  2 root root 4096 Sep 15  2021 games
drwxr-xr-x  2 root root 4096 Sep 15  2021 include
drwxr-xr-x  5 root root 4096 Oct 21 19:18 lib
lrwxrwxrwx  1 root root    9 Sep 23 17:17 man -> share/man
drwxr-xr-x  2 root root 4096 Sep 15  2021 sbin
drwxr-xr-x  6 root root 4096 Sep 15  2021 share
drwxr-xr-x  2 root root 4096 Sep 15  2021 src

I have tried launching NSIGHT Compute v2019.5.3 and NSIGHT Compute v2022.2.1 and both return the ‘unsupported GPU’ error.

Interestingly, NSIGHT Systems v2021.5.4 is able to remotely profile just fine.

Thank you for your time.

[nickjeliopoulosdev]

I stumbled upon this post Nsight Compute error:"Profiling is not supported on device 0" - #5 by christopher.friis.berntse that states Pascal GPU support was dropped for NSIGHT Compute 2019.5.3. Is this correct information?

[AastaLLL]

Hi,

For TX2, could you try if nvprof can meet your requirement?
You can find it in the below folder:

$ /usr/local/cuda-10.2/bin/nvprof -hnvidia@tegra-ubuntu:/usr/local/cuda-10.2/tools$ ls
Usage: nvprof [options] [application] [application-arguments]
Options:
       --aggregate-mode <on|off>
                        Turn on/off aggregate mode for events and metrics specified
                        by subsequent "--events" and "--metrics" options. Those
                        event/metric values will be collected for each domain instance,
                        instead of the whole device. Allowed values:
                        	on - turn on aggregate mode (default)
                        	off - turn off aggregate mode

       --analysis-metrics
                        Collect profiling data that can be imported to Visual Profiler's
                        "analysis" mode. Note: Use "--export-profile" to specify
                        an export file.

...

Thanks.

[nickjeliopoulosdev]

I have been able to use nvprof for most of my needs so far, thank you for the suggestion.