Python Script Hangs with Nsight Systems Profiling When NVTX is Enabled

Developer Tools Nsight Systems Profiling Linux Targets
1

Issue: Python Script Hangs with Nsight Systems Profiling When NVTX is Enabled

Description:

A minimal Python script (run.py) that uses subprocess.check_output hangs indefinitely when profiled with nsys and NVTX tracing is enabled.

Python Script (run.py):

import os
import subprocess
import sys

print(subprocess.check_output(['file', '-b', os.path.realpath(sys.executable)]))

Profiling Command (Hangs):

nsys profile -s none -t nvtx,cuda -f true -o demo python run.py

Observation and Non-Hanging Command

The hanging behavior is directly tied to the inclusion of NVTX tracing. When NVTX tracing is removed from the profiling command, the script executes, and the profiling completes successfully.

Non-Hanging Profiling Command (NVTX Trace Removed):

nsys profile -s none -t cuda -f true -o demo python run.py

Environment Details

Component Version/Detail
Operating System Linux-6.17.0-2-default-x86_64
Python Version 3.12.11 (Anaconda)
Nsight Systems CLI 2025.5.1.121-255136380782v0
NVIDIA Driver 580.95.05 (Reports CUDA Version 13.0)
GPU NVIDIA GeForce RTX 3070 Laptop GPU (8GiB)
1 Like
2

@Guy_Sz can you help with this?

3

@jie-ren thank you for reporting this issue.

I couldn’t reproduce it on my local system. Is there anything unusual with your Python interpreter? Or OS distribution?

I’m puzzled with your findings, because from what I can tell, run.pydoesn’t generate any NVTX events.

I suggest setting Python’s multiprocessing start method to “spawn” (the default is “fork”), and check if the issue persists. This is done by adding the following lines to your script:

import multiprocessing
multiprocessing.set_start_method("spawn")
4

It looks like the hang might not just be about NVTX itself, but also system resource limits. CPU, RAM, or concurrent processes can act as hidden bottlenecks during profiling. Before diving too deep into Nsight settings, I’ve found it useful to first detect potential bottlenecks using a tool like this. Sometimes what looks like a “hang” is just the system reaching its limits