nvcc error: 'ptxas' died due to signal 11 (Invalid memory reference) on goto statement

Accelerated Computing CUDA CUDA Programming and Performance
1

Hello,

When building my program I got: nvcc error : ‘ptxas’ died due to signal 11 (Invalid memory reference)

By removing all code and subsequently adding the code in blocks, I have tracked down the point of error at a goto statement within an if-statement section:

if (shared_memory_struct->var_x == true)
{
goto point_x;
}

If I place the same goto statement immediately before or immediately after the if statement, I do not get an error and everything builds fine

If I replace the if statement variable with another one, or simply with if (true), I still get the error

It is as if cuda does not want the goto statement within the if section

Why ever would this be?

CUDA Driver Version / Runtime Version 6.0 / 6.0
Nsight Eclipse

2

Within the function that the goto statement resides in, I have used a number of goto statements, and more than one goto statement (at different locations) jump to the point/ label that the particular goto statement that registers the cuda error, jumps to

Now, if I create an additional label just after the original label that I wish to jump to, such that both essentially point to the same location, but that the goto statement with the error does not share a label with any other goto statement, the program builds without error

3

It would be best if you could provide a short, self-contained complete example that someone else could copy, paste, and compile, to demonstrate the issue.

4

I would rather not provide the original code; it is lengthy and it may take time to follow

So, let’s try the following first, and see if you find it sufficient

I initially had:

function
{
     if (condition_W)
     {
          goto point_A;
     }

     [code]

point_A:

[code]

[code]

if (condition_Q)
     {
          goto point_A;
     }

[code]

[code]

if (condition_R)
     {
          goto point_A;
     }

[code]
}

The very first goto statement (pertaining to condition_W) had the error
The workaround seems to be this:

function
{
     if (condition_W)
     {
          goto point_A1;
     }

     [code]

point_A:
point_A1:

[code]

[code]

if (condition_Q)
     {
          goto point_A;
     }

[code]

[code]

if (condition_R)
     {
          goto point_A;
     }

[code]
}

I suspect one can likely replicate this by having multiple goto’s (I believe at least 3) in a function, all jumping to the same point

5

The message: ‘ptxas’ died due to signal 11 (Invalid memory reference), indicates an internal compiler error. The compiler should not segfault no matter what code is passed to it. It would be helpful if you could file a bug using the bug reporting form linked from the registered developer website, and attaching a self-contained repro code. Since the problem occurs in ptxas which compiles PTX to SASS (machine code), submitting the PTX code should be sufficient should you not want to submit the actual CUDA code. In this case, please note the exact ptxas commandline in the bug report. Thank you for your help.

6

I thought that one could replicate the issue by having multiple goto’s in a function, all jumping to the same point; but this seems not to be the case - the test program compiles fine

I have now stripped the code leg with the problem from the grand program, and have also stripped the code of the underlying functions

The problem persists in this copy of the program, hence you can use this to trace the issue

7

How does one observe the problem? After downloading and unzipping your tar archive, what commands should I use?

Is it possible for you to distill the example down to a single file?

8

It is a nsight eclipse project archive; hence, you can simply import the project by means of the archive file
You would then have a full-fledged project
When you subsequently build the the project, you would experience the error; I have made sure that the project contains the error when building

9

I tried importing the test3 project at the top level of the directory structure in the project. It doesn’t seem to be working. I tried build project, but I get the error, “no rule to make target all”. I also tried importing at the Debug directory, but got another error when I tried to build the project that way.

I’d suggest either providing detailed steps to demonstrate the issue (unzip, move to directory…, launch nsight EE, select import… etc.), or condensing the repro to a single file and provide the compile command for that file. If you’re building a project in nsight, only one file is getting compiled at a time anyway. You should be able to extract just that file, and the build command from nsight EE, and provide that.

10

nsight eclipse: new cuda c/c++ project (empty project); name the new project; select the new project - right-click and select import; select archive file; select the actual archive file (stored somewhere) you downloaded; expand test3 as project; select everything except the debug folder to import; make sure that “place in folder” = the new project folder
Importing is normally a mess in my experience; you should note the test3 project now as a sub-folder under the new project
Expand the test3 project folder and drag the 2 header folders (prefixed H_), the 1 source folder (prefixed S_) and the 1 source file (main.cu) from under the test3 folder and drop them such that they reside immediately under the new project folder
You can now delete the test3 folder under the new project folder
Add the mentioned header folders to project → paths and symbols; add the source folder to project-> source location; enable separate compilation; select (only) compute capability = 3.5
Build the project; experience the error - I myself have done this moments ago and it is there

11

so rather than that complicated set of instructions, could you identify that file that is being compiled at the time the error is thrown, and then grab the compile command that generates that error? I think this is a relatively trivial copy-and-paste from what is shown in the nsight console window.

12

The console output when building the project contained in the project archive file:

06:33:05 **** Incremental Build of configuration Debug for project test3 ****
make all
Building file: …/S_group1/G1c_filter_krnl_v2_1.cu
Invoking: NVCC Compiler
/usr/local/cuda-6.0/bin/nvcc -I"/home/…/cuda-workspace4/test3/H_group1" -I"/home/…/cuda-workspace4/test3/H_group5_v2" -G -g -O0 -gencode arch=compute_35,code=sm_35 -odir “S_group1” -M -o “S_group1/G1c_filter_krnl_v2_1.d” “…/S_group1/G1c_filter_krnl_v2_1.cu”
/usr/local/cuda-6.0/bin/nvcc --device-c -G -I"/home/…/cuda-workspace4/test3/H_group1" -I"/home/…/cuda-workspace4/test3/H_group5_v2" -O0 -g -gencode arch=compute_35,code=compute_35 -gencode arch=compute_35,code=sm_35 -x cu -o “S_group1/G1c_filter_krnl_v2_1.o” “…/S_group1/G1c_filter_krnl_v2_1.cu”
nvcc error : ‘ptxas’ died due to signal 11 (Invalid memory reference)
nvcc error : ‘ptxas’ core dumped
make: *** [S_group1/G1c_filter_krnl_v2_1.o] Error 139

06:33:07 Build Finished (took 2s.66ms)

Does this suffice?

13

Yes, that helps. I am able to reproduce the error. Thanks. As a temporary workaround, I think you will have better results if you build the release version of the project instead of the debug version of the project in nsight EE. (removal of the -G switch in that particular compile command seems to eliminate the error for me.) I understand this has implications for debugging. I merely point it out as an observation.

14

Noted

I am in the middle of debugging the particular project though; hence, this is something I would want to avoid doing if possible

Even though the compiler does not explicitly state a code line that causes or ‘helps’ it to seize compiling, I managed to isolate a single goto statement, as mentioned
If I isolate this particular goto statement, the project builds fine, and I can debug normally

15

The file that is being compiled when the error occurs is G1c_filter_krnl_v2_1.cu

Could you be specific about which line in that file you isolated the difficulty to?
I wasn’t able to figure that out by reading this thread, although I do see that you mention isolating it to a goto statement.

I’ve filed a bug with nvidia for this issue. The bug number is 1509115 for reference.

16

The very first goto statement at the very beginning of the filter_main_loop function; I believe it is line 404

If you move that particular goto out of the if section - above it, or below it - it should build fine

Also, that particular goto statement jumps to the label ‘point_F’; if you create an additional label just below ‘point_F’, and have the goto statement jump to it instead, it should build fine too

Like so:

point_F:
point_F1:

[code]

17

Thanks! I’ve duplicated that as well, and added a note to the bug.

18

I have tested, and this particular issue appears to be fixed in CUDA 6.5 RC which is now available to registered developers.