Broken Backtraces: Nsight systems on Jetson Xavier NX

I would like to profile a (C++, ROS) application on the Jetson NX using Nsight Systems but I can’t resolve an issue with brocken backtraces for the CPU processes. I’ll appreciate any help:

system
Target: Jetson Xavier NX with JetPack 4.5.1
Target Nsight Systems CLI version: 2020.5.3
Host: Ubuntu 20.04
Host Nsight Systems w/ GUI version: 2020.5.1 (2020.5.3 does not seem to be available).

Problem:
When visualizing in nsys-ui on the host the .qdrep file that was generated on the target, I do get brocken Backtracesin the top-down view.
The steps that I did before generating the .qdrep file were (according to the user guide):
sudo sh -c 'echo 2 >/proc/sys/kernel/perf_event_paranoid'
and use the build flags:
catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-g -funwind-tables -fno-omit-frame-pointer" -O2
My status of nsys status -e reads:

Sampling Environment Check
Linux Kernel Paranoid Level = 2: OK
Linux Distribution = Ubuntu
Linux Kernel Version = 4.9.201-tegra: OK
Linux perf_event_open syscall available: OK
Sampling trigger event available: OK
Intel(c) Last Branch Record support: Not Available
Root privileges: No  # <<-- note that I use "sudo nsys profile script.sh" instead
Kernel module: Available
Sampling Environment: OK

The profiling itself was launch like this:
sudo nsys profile -d 45 --process-scope=process-tree --backtrace=fp bash profile_script.shI also tried without --backtrace and with --backtrace=dwarf. The backtraces stay broken.

What can I do to solve this?

Not sure if this is related: I installed nsights systems not via the SDK Manger (would need to reflash but want to avoid to set up my system again), but via apt: sudo apt install nsight-systems-cli-2020.5.3
(for jetpack nsight 2020.5 seems to be the latest version as of now).

Have you taken a look through Nsight Systems User Guide :: Nsight Systems Documentation Backtraces?

thank you for the quick response!

yes, I did add the compiler flags listed in the mentioned section under ARM:
-g -funwind-tables -fno-omit-frame-pointer

could the part Root privileges: No lead to the broken backtraces? (even though I’m running the profiling with sudo)

@Andrey_Trachenko , can you help with this?

@eeppii,

Hi Thomas,

2020.5.XXXX releases - there is regression for NSys CLI on Tegra - unknown modules/unresolved symbols/broken backtraces for some modules.

NSys releases:

2020.4.XXXX - the issue is not reproduced
2020.5.XXXX - reproduced in CLI (profiling via GUI should be OK)
2021.X.XXXX - not reproduced

Please try some 2021.X.XXXX release (it should be compatible with JetPack 4.5.1), for example 2021.1.XXXX or 2021.2.XXXX.

Link for 2021.1 release (verified on L4T Jetson Xavier):
https://developer.nvidia.com/rdp/assets/nsight-systems-2021.1-aarch64-cli-only-deb-installer

–backtrace=fp
I also tried without --backtrace and with --backtrace=dwarf.

--backtrace=auto option is actually enabled by default, auto includes FP and DWARF unwindings.
So, the following commands are equal (NSys will use the best backtracing algorithm for each stack frame):
sudo nsys profile <application>
sudo nsys profile --backtrace=auto <application>

-g -funwind-tables -fno-omit-frame-pointer

yes, it looks good

perf_event_paranoid

by default, NSys ignores this on Tegra platforms

I use “sudo nsys profile script.sh”

yes, sudo is needed for sampling on L4T

Thanks,
Igor.

1 Like

That worked! Thank you! :)
(used version 2021.1.1 on host linux and jetson nx)

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.