Generating separate cubin files

(Using Visual Studio 2015)
I would like to generate “.cubin” files, and, store them in an other place instead of in the executable.

I tried in the vcproj properties, CUDA C/C++, Common Options, NVCC Compilation Type to set to
“Generate .cubin file (-cubin)”, but that did not work for me.

Help appreciated
Chris

Is your ‘Compiler Output’ property above ‘NVCC Compilation Type’ still set to the default ‘$(IntDir)%(Filename)%(Extension).obj’ value? It might be that it’s generating a .cubin-format file but it’s named .obj due to this setting.

‘NVCC Compilation Type’ is set to ‘Generate .cubin file (-cubin)’
'Compiler output (obj/cubin) is set to ‘(IntDir)%(Filename)%(Extension).cubin' (Cuda Linker) 'Linker Output is set '(IntDir)$(TargetName).device-link.obj’

I can’t find any ‘.cubin’ or '.device-link.obj’ file anywhere I looked

These steps don’t reproduce for me, I do get a .cubin file. Can you please provide a small sample project that reproduces what you’re seeing? If so I’ll take a look and see what’s going on. Thanks.

Thanks for your offer

I made a copy of the sample program vectorAddDrv.
I deleted all files not in the top directory.
I changed the project properties like posted above (VS2015 pro)
I clicked on “rebuild”

I found in
data\
a file vectorAdd_kernel64.ptx
This is a text file and it looks like PTX to me (Assembler like instructions)

I found in
x64\debug\
a file vectorAdd_kernel.cu.cache
This is a text file; each line looks like a finished or unfinished assignment

And for the fun of it, I ran the program. I didn’t expect that, but it wrote “PASS”

You can ignore the .cache file, that’s an implementation detail of the VC++ build customization. In a nutshell, it’s the build properties of the source file when it was built, it’s used to determine if CUDA files need to be rebuilt when the project file changes.

To clarify, you set ‘NVCC Compilation Type’ to ‘Generate .cubin file (-cubin)’ and ‘Compiler Output (obj/cubin)’ to ‘$(IntDir)%(Filename)%(Extension).cubin’ but when you re-built the output was a .ptx file and there was no .cubin? If so, it seems like it is building with the same settings as the original sample. If you look at vectorAddDrv_vs2015.vcxproj, what do you see for vectorAdd_kernel.cu?

You are right, here the word ‘ptx’ occurs twice



data/%(Filename)64.ptx
ptx

I have tried to change ‘ptx’ to ‘cubin’ in the .vcproj with a text editor:

-If I changed it in the ‘CompileOut’ line, the file name for the generated output was changed to ‘cubin’.
However, the content of that file was still ptx (identical to before).

-If changed it in the ‘NvccCompilation’ line, the build fails with an error message:
nvcc fatal : Option ‘–cubin’ is not allowed when compiling for a virtual compute architecture

CompileOut is just a filename, it’s going to name the file whatever is specified here regardless of how it’s compiled, so that’s expected behavior. It sounds like it is trying to compile to cubin after the NvccCompilation change. Did the nvcc error message help you figure things out?

It compiled to PTX. No, the error message did not help me, as I found in the documentation that --cubin would be a legal argument.

What setting do you have for Device/Code Generation? From the error message I’m guessing that it’s only specifying virtual compute architecture. Try something like ‘compute_35,sm_35’, does this give a different result?

YES, that made a difference.
Thank you for your perseverance.
Chris

Sorry to come back to my question, but:

The solution up to the previous reply did work, but my understanding does not.

In short:
-I have this “nice” vcxproj file which generates a working .obj file, which contains both, a host section and cubin section.
-I want to change that vcxproj file, so that it generates two output files instead:

  1. a working host-side only object file
  2. a separate cubin file, usable for cuModuleLoad .

I found this documentation in NVCC doc (3.2.1):
-cubin: Compile all .cu/.ptx input files to device-only .cubin files. This step discards the host code for each .cu input file.
(Emphasis mine)
Which could be an answer for half the question; I still need the other half.

If I understand correctly, you need to build the same source file twice with nvcc but with different settings. Is that correct?

I build binary files. I need two binary output files; one with the binary X64 host code and one with the binary cubin code for the GPU. Whether I achieve this with one compilation and perfect flags, or I achieve this with two compilation steps is of lesser importance.

Thanks,
Chris

Is it an option to move the host code out of the .cu file and into a .cpp file? Or you’re looking to have the exact same code compiled both ways?

That is an interesting question. The answer of which I am not really sure of (that would be up to my customer). I don’t want to face the risk of source files not fitting together, neither would I like to change all source files.

I remember having read that no matter what, somewhere there is X64 code which calls the CUDA drivers to load the cubin code into the GPU. I do not separate my code in what compiler created it, but according to where it executes. (I have some fancy post-processing steps which are machine dependent)

OK. I could offer some suggestions, but I think this would be a better question for the CUDA Programming forum, they would know best.

https://devtalk.nvidia.com/default/board/57/cuda-programming-and-performance/

thanks for your help; I am continuing the disussion on
https://devtalk.nvidia.com/default/topic/1046884/nsight-visual-studio-edition/generating-separate-cubin-files/
https://devtalk.nvidia.com/default/topic/1047461/cuda-programming-and-performance/split-x64-and-cubin-into-separate-files-/