Running CUDA Visual Profiler

Accelerated Computing CUDA CUDA Programming and Performance
1

Hi everyone,

I know that this subject has been extremely discussed in the last few months by the all posts I have read, but I still can’t find any good solution to my problem, which is pretty simple: I can’t get CUDA Visual profiler to work with my own CUDA programs.

I’m working on Linux 64 bits (Fedora Core 8) using as a graphics card a GeForce 8600.

When I use CUDA Visual Profiler with the examples given working with the projects of the SDK, it works just fine, but with mine, it fails and I get the following message :
Error -94 in reading profiler output.
Minimum expected columns (method,gputime,cputime,occupancy) not found in profiler output file.

I’m pretty sure it has something to do with the environment variables CUDA_PROFILE, CUDA_PROFILE_LOG, etc. as written in the CUDA_Profiler_2.0.txt (that I have read carefully) but everything I’ve tried so far to set those variables has failed up to now,

As I have read, I’ve tried running the program from the console writing down “export CUDA_PROFILE=1” before both building and running my program and before launching CUDA Visual Profiler but with no more result than the Error -94.

So the question is simple : What is wrong and what did I miss while I tried to enable my cuda profiler ?

Thx in advance and sorry for this question that will probably seem really green to the majority of the readers,

J.

2

You should make sure that the current directory for your program when running your kernels is the directory you set in the visual profiler settings.

I remember having a matlab program that does initialize; cd subdir; run cuda-mex file; exit;

I had to set ~/subdir as start-in directory in the visual profiler and change my script into : cd …; initialize; cd subdir; run cuda-mex file; exit;

This is all from memory, but I hope it helps you. If it still does not work beginning of september send me a pm, I’ll be back from holidays & NVISION2008 beginning of september.

3

I already checked, but, no, it still doesn’t work. Here’s what I tried :

from the shell terminal :

export CUDA_PROFILE=1

export CUDA_PROFILE_CSV=1

Then I launched the Visual Profiler. I set the target which is my executable. The workspace is the same as the one where my .exe is and I even tried to set the workspace as the one where my CUDA files are, but with no more results than the annoying Error -94 External Media

Did I need to set a CUDA_PROFILE_CONFIG file? If so I don’t understand why for I have read that one doesn’t need it while working with the VISUAL Profiler and not doing it manually…

Well, I’m beginning to be out of ideas on how to set the environment variables to set the Cuda Profiler on. Any thoughts would be gladly appreciated !

Jerome

4

Would have been nice to know this before, but here it is:

I wrote in my /etc/profile : export CUDA_PROFILE=1 and restarted, I now see a cuda_profile.log, which I didn’t see yesterday. First step done

In the cuda_profile.log I now see my kernel(s) as this:

method,gputime,cputime,occupancy
method=[ __globfunc__Z17kernelInvertColorPiS_ii ] gputime=[ 4566.848 ] cputime=[ 21.000 ] occupancy=[ 1.000 ]
method=[ __globfunc__Z17kernelInvertColorPiS_ii ] gputime=[ 4565.888 ] cputime=[ 18.000 ] occupancy=[ 1.000 ]
method=[ __globfunc__Z17kernelInvertColorPiS_ii ] gputime=[ 4565.056 ] cputime=[ 18.000 ] occupancy=[ 1.000 ]
method=[ __globfunc__Z17kernelInvertColorPiS_ii ] gputime=[ 4569.056 ] cputime=[ 18.000 ] occupancy=[ 1.000 ]

It gives me also this when using export CUDA_PROFILE_CSV=1 in a terminal window before launching the program:
method,gputime,cputime,occupancy
__globfunc__Z17kernelInvertColorPiS_ii,4576.352,18.000,1.000
__globfunc__Z17kernelInvertColorPiS_ii,4567.104,17.000,1.000
__globfunc__Z17kernelInvertColorPiS_ii,4565.824,17.000,1.000
__globfunc__Z17kernelInvertColorPiS_ii,4564.960,21.000,1.000

I thought it would work fine, but guess what, it’s still giving me the error -94 (as mention above).

I’m making sure that the name of the .exe and the name of the working directory, where my cuda_profile.log is located, are O.K. in the Session settings.

Is this has something to do with the CUDA_PROFILE_CONFIG? I don’t know what to do about it because all the options mentioned in the CUDA_Profiler_2.0.txt are available in the visual Profiler so I think (correct me if I’m wrong) that I don’t need to set up this file since I’m doing it through Visual Profiler.

I find it hard to believe that nobody has an idea about this while so much people seems to use the Visual Profiler and, most of all, make it works.

Any help, other than mine, would be greatly appreciated.

Thank you,

Jerome

5

Hopefully, i’m working “on the side” about this… otherwise I’d feel like I’m wasting my time.

I’ve set in /etc/profile the folloqing command :
export CUDA_PROFILE_CONFIG=$HOME/.cuda_prof_config

in which I set the parameters that I want the cuda profiler to analyze.

In this example, I’ve set the following :
gld_coherent
gld_incoherent
gst_coherent
instructions

and I can see the following in my cuda_profile.log :

method,gputime,cputime,occupancy,gld_coherent,gld_incoherent,gst_coherent,instructions
method=[ __globfunc__Z17kernelInvertColorPiii ] gputime=[ 2866.208 ] cputime=[ 3006.000 ] occupancy=[ 1.000 ] gld_coherent=[ 49152 ] gld_incoherent=[ 0 ] gst_coherent=[ 196608 ] instructions=[ 358760 ]
method=[ __globfunc__Z17kernelInvertColorPiii ] gputime=[ 2871.296 ] cputime=[ 2992.000 ] occupancy=[ 1.000 ] gld_coherent=[ 49152 ] gld_incoherent=[ 0 ] gst_coherent=[ 196608 ] instructions=[ 358624 ]
method=[ __globfunc__Z17kernelInvertColorPiii ] gputime=[ 2848.512 ] cputime=[ 2969.000 ] occupancy=[ 1.000 ] gld_coherent=[ 49152 ] gld_incoherent=[ 0 ] gst_coherent=[ 196608 ] instructions=[ 355350 ]

That’s good, I got all the info that I needed, but what is the matter with the Visual Profiler ?

I still got the Error - 94 in reading profiler output
Minimum expected columns…

I don’t understand because I thought that the profiler output was cuda_profile.log. Isn’t the case ? It seems like I got all the needed columns as you can see…

The only thing missing here seems to actually see the “visual” cuda_profile.log in the Visual Profiler. Has anyone been successful in that matter ?

Thx,

Jerome

6

all the settings you are doing only affect the normal profiler, not the visual profiler. For all settings of the visual profiler, you should be making them in tghe visual profiler.

7

Finally got it

I went the other around when I should have stayed with the visual profiler setup. Anyway I got both the Visual Profiler and the normal profiler working now…

What I first did is importing a .csv file from the examples given with the CUDA Visual Profiler, it finally worked with my program, and then I did a “save as” and voilà !

I don’t know (or don’t care to know) why it isn’t possible to create a .csv file from the start with the Visual Profiler, I didn’t see anything related to this in the documentation.

Anyway, just tried to help if someone is googling the same problem as I had.

Jerome

8

HI…

I’m also stuck with the same Error -64 Problem…

Can you explain in bit more detail … how you solved this problem

Thanks

9

HI…

I’m also stuck with the same Error -64 Problem…

Can you explain in bit more detail … how you solved this problem

Thanks