Visual Profiler not working (Win XP 64 bit) getting errors related to the profiler output

I have the 32 bit CUDA toolkit 2.0 and Visual C++ 2005 Express Edition installed on Windows XP Professional 64 bit.

I downloaded the Visual Profiler 1.0 and when I run it with one of my projects, I get

“Error -94 in reading profiler output. Minimum expected columns (method, gputime, cputime, occupancy) not found in profiler output file”.

When trying the Visual Profiler 1.1, this complains about the header line of the CSV being empty.

What could be the culprit here?

NVIDIA drivers are at version 6.14.11.8070.

Christian

I also have this issue with XP 64-bit but running the 64 bit CUDA Toolkit 2.1 and Visual C++ 2008 full. If I run my program with a modest data set it Profiles fine, however if I run it with a full data set then the Profiler fails after my program has run with the error “Empty header line found in CSV file.”. This happens at the end of a single run, and at the end of multiple runs. It also occurs regardless of the session settings I have set.

Cheers,
John

I am having this same problem. It only happens when I check one of the profiler counters or “other options.” For some of the options, the temp_cuda_profiler.conf will have something like this: “Error in profiler config file: /Users/arilee/Desktop/dft_test/temp_cuda_profiler.conf
Invalid profiler config option: gridsize” Any ideas? :(

I forgot to mention that I’m running it on a mac.

use the new version… version:1.1.08

I have already tried version:1.1.08, it didn’t work either.
The problem I encountered was exactly like RoofusGreen’s .I run it on Windows XP 32-bit
So, anyone has a solution to this? That will be grateful!

I think I know why this may be occuring. In the Visual Profiler Session Settings the Max Execution time is set by default to 30 seconds. If the profiler runs multiple sessions it seems not to terminate the target application sucessfully.

My application uses OpenGL and is based on the simple example in the SDK. Consequently deinit processes are only called when the ESC key is pressed. Visual Profiler doesn’t terminate the target application by sending an ESC, so after several runs CUDA crashes.

If I manually terminate the target application before the Max Execution time for each run of the application then Visual Profiler works fine. If I let the timeout happen then it will always fail as described above.

Note I am running glew64.lib and a modified glut.h that uses glut64.lib.

Hope this helps.

I tried your suggestion, but it still fails. :( It fails on tests that only require 1 run as well.

I too had the problem with -94 Error:

“Error -94 in reading profiler output. Minimum expected columns (method, gputime, cputime, occupancy) not found in profiler output file”.

But I partially solved the problem. Upon inspection of my working directory I found a file ‘temp_cuda_profiler_0.csv’ which actually contained the profiler data. The problem turned out to be that it also contained these first two lines:

CUDA_PROFILE_LOG_VERSION 1.3

CUDA_PROFILE_CSV 1

As it was mentioned on some previous post, these lines can cause trouble. So I removed them and imported the altered file .csv file into Visual profiler. The import was successful, and I was able to see my results. Moreover, when I run the session again, it work properly (no -94 Error).

However, I still have a problem if I set some additional Session setting features (e.g. if I check Kernel Options or Timestamp). In this case in the ‘temp_cuda_profiler_0.csv’ file contains no data, except this text:

Error in profiler config file: D:/MyDocs/Research/CUDA/MatrixInversion/temp_cuda_profiler.conf
Failed to open profile config file.

Indeed, there is no ‘temp_cuda_profiler.conf’ in my working directory. So I tried creating one, setting profiler options ‘manually’. However, result of a profiler session run was the same (the same ‘Failed to open config file’ error), and the .conf file I created was deleted. So it looks like the Visual profiler deletes its own config file, and then complains it cannot read it ;o)

I am running on WinXPx64 with Visual Profiler 1.1.08 and CUDA Toolkit 2.1

Cheers,

Radoslav

I am facing the same problem. In spite of the error “Error -94 in reading profiler output. Minimum expected columns (method,gputime,cputime,occupancy) not found in profiler output file.” I do get only these four columns in my .csv file. I can use this file well in the Visual Profiler by removing the above two lines. But the moment I check “Timestamp” or any four (max we can enable at a time) or less of the “profiler counters”, I get this in all the three .csv files "Error in profiler config file: C:/Documents and Settings/arr206/Desktop/Aditi-work/Double Precision Final/05_CSDD_GPU_double_tiles_rectangular image_optimized_01/temp_cuda_profiler.conf

Failed to open profile config file."

Please note that my arguments to Visual Profiler are “-jvm -nosplash -r test”. CUDA_PROFILE and CUDA_PROFILE_CSV are enabled (rechecked using cmd…so I am sure about this), Max. Execution Time is set to 2000secs. I tried playing around by setting the variable CUDA_PROFILE_LOG value to the current project directory and current_project_directory/temp_cuda_profiler.conf but did not help.

There needs to be this temp_cuda_profiler.conf file with max 4 profile counters enabled at any time. Has anyone been able to set this problem right? Will be thankful if someone could suggest something.

Thanks & regards,

Aditi

Yippeee…I got the solution!! :D

I went through various posts discussing the same topic and after lot of hit-and-trial finally got it running!! Following are the tips for others who may stumble on this thread looking for an answer and feeling the same miserable as I was a while ago!!

With Profiler 1.0

  1. CUDA_PROFILE and CUDA_PROFILE_CSV set to 1 in controlpanel->system->advanced->environment_variables (in both the boxes).
  2. Run the cuda project using visual profiler. For that, set the following in the Visual Profiler->Profile->Session Settings:
    a) Launch: “C:/Program Files/MATLAB/R2008a/bin/win32/MATLAB.exe” (I figured this out after a long time that I was supposed to use this and not C:/Program Files/MATLAB/R2008a/bin/MATLAB.exe …win32 made all the difference!!)
    cool.gif Working Directory: “Your cuda project directory”
    c) Arguments: “–noprompt” OR "–noprompt -r <name of the .m file to run without .m extension>
    d) Max. Execution Time: (Set it enough long) “2000secs”
  3. In the Configuration: Enable “Time Stamp” and only a max of 4 “Profile Counters” at anytime. So do it in three runs for all counters.
  4. Ignore the Error -94.

With Profiler 1.1 and onwards:

  1. Same
  2. a) Same
  3. cool.gif Same
  4. c) -r <name of the .m file to run without .m extension>
  5. Check everything -> all parameters, time stamps, counters, extra parameters, etc. All are available in one go.
  6. Doesn’t appear
  7. Keep closing every Matlab session as it ends. If you wait for it to happen automatically, that never happens.

I was earlier skeptical that Profiler 1.1 would not work with cuda2.0 and I was not ready to move to higher versions of CUDA and re-build the familiarity…but it did work!!

BOOM!! You have all the results!!

I still have the same problem, and the magic formula above unfortunately doesn’t help :(
Freshly installed machine with the latest Windows XP driver 32-bit, CUDA sdk & toolkit 2.2.

The profiler works with CUDA SDK samples such as template and reduction, but with my project it goes crazy.

Actually it does run the project correctly (I can see the correct output), but after that it pops out a dialog box saying that it’s running the program. This takes ages to finish, and afterwards I just get a message:

Program run #1 was aborted after maximum program execution time duration of 100 seconds.
Empty header line found in CSV file.

The one thing that’s a bit special about my project is that it checks the command line arguments, and has two different behaviors when there are no args and when there are some arguments. The second different thing, is that i used this time a CUDA VS Wizard to build the project.

Anyways it runs correctly and fast in normal release, debug, etc modes, when I do not attempt to do any profiling.

Can anybody PLEASE help to resolve this problem quickly?

p.s. I tried setting the -noprompt, empty space, and correct arguments but the behaviour is the same…

the profiler output is sometimes empty file (0 bytes), sometimes like this:

cuda_profile.log:

CUDA_PROFILE_LOG_VERSION 1.4

CUDA_DEVICE 0 GeForce GTX 280

CUDA_PROFILE_CSV 1

method,gputime,cputime,occupancy
memcopy,194.624,1109.942
memcopy,388.768,614.264
Z21kernel_1PjPKjS1_S,386.048,4.853,1.000

there isn’t any .csv file for my project!

Make sure your program terminates within the maximum execution time.

Make sure you call cudaThreadExit() on termination of your program.

Maybe this helps (haven’t tested it myself yet, but some other forum posting suggested this).

Christian

Christian, thanks x 1000! cudaThreadExit() resolved the problem!!! :)

I’ve encountered a couple applications that won’t run on the 185.85 release. OpenGL Extensions Viewer 3.12 rendering tests fail at the 1.3. If I drop back to the 181.20 driver package, it successfully run through the complete testing range (1.1 - 2.1). Of course this could be a problem with the application.

The second program that fails is Starry Night 6 Pro Plus. This program uses openGL extensively and again, dropping back to 181.20 fixes that problem.

I’m running winXP x64, on a AMD 4800+ x2, 4gb memory, and a 9800GT video board.

Thanks Christian - that was exactly the problem I had! :))
Now on to optimizations …

Hi.

I’ve had the same problem (Empty header line found in CSV file.) It occured when I had max execution time set to 30. The error was displaying no matter if I set multi or single run. The solution in my case was to change max execution time (to longer). When set to 60 or 90 seconds it works fine. I don’t know why it works with longer time, and why not with shorter time, but it solved my problem and that was enough for me.

eMeS

oh…now i have also the same problem.

the cudaThreadExit() is called on termination of my program and i|ve charge a time. it still doesnt work…

Empty header line found in CSV file.

Error in reading profiler output.

:pinch:

I have this problem as well. I have downloaded the CUDA 3.0 tookit

I execute a matlab script with the “-nojvm -nosplash -r cudaScript” parameter set. The cudaScript.m contains the matlabcode for calling the mex-file, which has a cudaThreadExit and all.

Using the GUI, it starts 4 threads all running Matlab (can you reduce this to 1 somehow?). Basically, it gives me the error with the temp_cuda_profiler.conf file missing, and that it can’t read the profiler output. This happens even before the Matlab threads finish (using an exit command in matlab).

Also, I can’t get it to work using the green PLAY button. If I press this, the program crashes without ever doing anything. By pressing the Session settings button and then pressing the Start button from there allows me to start everything.

It seems this tool is somewhat unstable still…

Anybody having these issues?

Cheers