Nvprof - Unified Memory profiling failed [solved]

Development Tools Other Tools Visual Profiler and nvprof
#1

Hi all,

I’m trying to profile my program which is using UM. Unfortunately, I cannot profile the memory copy process…

When I add the flag “–unified-memory-profiling per-process-device” I get the Error:
Error: unified memory profiling failed.

I am aware of the following posts:
https://stackoverflow.com/questions/39962675/unified-memory-profiling-failed?answertab=votes#tab-top
https://devtalk.nvidia.com/default/topic/1010924/visual-profiler/unified-memory-profiling-failed/post/5159265/#5159265
But I do not have root permissions on the server I am profiling on.

Is there a solution for this error?

I’m using CUDA 8.0 and a K40m on Ubuntu 16.04.4 LTS.

Thanks in advance,

Adrian

#2

Hi, admich

Before the error printed, have you seen any warning about uvm ?
I’m afraid you system do not meet uvm profiling system requirements. Then you get such error.

#3

Hi veraj,

Thanks for your reply!

No, I do not get a warning.

This is all I get when I execute my program:
/usr/local/cuda-8.0/bin/nvprof --unified-memory-profiling per-process-device ./build/mb_bandwidth_unified_v5
==3569== NVPROF is profiling process 3569, command: ./build/mb_bandwidth_unified_v5
======== Error: unified memory profiling failed.

I checked my system with the requirements listed here:
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-requirements

Since I’m using a x86_64 Ubuntu and a card with sm 3.5 it should work.

Other Ideas?

Best regards,
Adrian

#4

Hi, admich

As the devtalk link you provided before, we can not reproduce this issue on CUDA 8.0.

  1. Upgrade your toolkit/driver to latest version, for current stage, it is 9.1. 9.2 will out soon
  2. Check if it is sample specific, you can try SDK sample 0_Simple/UnifiedMemoryStreams to see if it works

If all not work, I’ll check with dev if we have this GPU support issue.

#5

[Solved]:

I faced the same situation as unified memory profiling failed, try adding the cuda path to the sudoers list in the following way,

1.sudo visudo
2.modify the defauslt secure path int he following way

Defaults        secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/usr/local/cuda/bin/:"

now try to run the command/application

sudo nvprof ./matmul
#6

Great. Thanks for the info.

#7

You don’t need root access to fix this. Since you are using Cuda 8.0 this should be working. The problem lies in the ‘iBUS’ config. All you have to do is delete the folder /home/[user]/.config/ibus/bus and the problem will be gone.

#8

Any specific reason for this issue related to bus?