Hi, I am running tensorflow benchmarks on a multinode multi GPUs system (2 nodes and 8 GPUs in toTAL). I need to profile the application and I am following the instructions shown in [MPS PROFILING WITH NVPROF][http://www.sie.es/wp-content/uploads/2015/12/cuda-profiling-tools.pdf]; however, Visual Profiler is launching the message: “Analysis is not supported for multi-process applications”
Some tips?
Thanks for the link, it shows the same instructions I am following but still getting the error “Analysis is not supported for multi-process applications” when trying to explore the guided and unguided modes. Also nvprof is generating only 4 data files instead of 8, even though in the command line I am passing this information (mpirun -np 8 -H 192.168.10.1:4,192.168.10.2:4)
If you are able to import the nvprof data into nvvp, and see the timelines, then that is probably what you will get.
I think the message “Analysis is not supported for multi-process applications” is correct.
You can import timelines, you may be able to look at metric data, but you will not be able to use guided analysis features.
You may need to turn off the “run guided analysis” checkbox when starting nvvp.
nvprof will only generate files for processes which launch on the machine that you are running it on. It will not capture files or data for processes on other machines.
Hi to all,
so there is no way to achieve guided or unguided analysis on multiprocesses? (with or without MPS)
Is any update of how to achieve it?
Thanks