Compiling Nvidia CUDA 10.1 in Visual Studio 2019 (Enterprise) project Fails

smukachi · May 6, 2020, 5:54pm

Hi all. This is my third day struggling with compiling a simple HelloCuda CUDA project in visual studio 2019 (Enterprise). I installed the latest Nvidia CUDA 10.1 Toolkit without errors. Please see the ‘Error List’. I cannot work out what this error means except that it appears the target (versions) between MSVC & CUDA Toolkit 10.1 might be mismatching and I really do not know what how to resolve that. These are the only errors I get.

Severity	Code	Description	Project	File	Line	Suppression State
Error	MSB3721	The command ""C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe" -gencode=arch=compute_35,code=\"sm_35,compute_35\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio019\Enterprise\VC\Tools\MSVC4.22.27905\bin\HostX86\x64" -x cu  -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include"  -G   --keep-dir x64\Debug -maxrregcount=0  --machine 64 --compile -cudart static  -g   -DWIN32 -DWIN64 -D_DEBUG -D_CONSOLE -D_MBCS -Xcompiler "/EHsc /W3 /nologo /Od /Fdx64\Debug\vc142.pdb /FS /Zi /RTC1 /MDd " -o x64\Debug\kernel.cu.obj "C:\Working\NVIDIA\HelloCuda\kernel.cu"" exited with code 1.	HelloCuda	C:\Program Files (x86)\Microsoft Visual Studio019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations\CUDA 10.1.targets	764

Thank you in advance for the your help.

Elton · February 1, 2020, 8:49am

Try copy/pasting the nvcc.exe command-line to a command prompt in your project directory, run it, and see if there is a more detailed warning/error message from nvcc.

smukachi · February 29, 2020, 1:30am

Hi Elton, thank you for an amazingly quick reply. If I understood your instructions right, here is what I did and what I got back.

C:\Working\NVIDIA\HelloCuda>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe" -gencode=arch=compute_35,code=\"sm_35,compute_35\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio019\Enterprise\VC\Tools\MSVC4.22.27905\bin\HostX86\x64" -x cu  -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include"  -G   --keep-dir x64\Debug -maxrregcount=0  --machine 64 --compile -cudart static  -g   -DWIN32 -DWIN64 -D_DEBUG -D_CONSOLE -D_MBCS -Xcompiler "/EHsc /W3 /nologo /Od /Fdx64\Debug\vc142.pdb /FS /Zi /RTC1 /MDd " -o x64\Debug\kernel.cu.obj "C:\Working\NVIDIA\HelloCuda\kernel.cu"
nvcc fatal   : Cannot find compiler 'cl.exe' in PATH

Thanks

Elton · February 1, 2020, 8:49am

Hmm is there a cl.exe under C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.22.27905\bin\HostX86\x64 on your machine?

Another thing to check is run the command-line again while running Process Monitor (https://docs.microsoft.com/en-us/sysinternals/downloads/procmon). Filter for only file activity, for when nvcc.exe is the process, and the path contains cl.exe, and see where all it’s looking for cl.exe.

One more option to help diagnose it, run msbuild /v:diag [project].vcxproj in the command-line, redirect the output to a file (it will be very verbose), and see how the build is configuring the path in the log file.

smukachi · February 1, 2020, 8:49am

Yes, cl.exe is in the C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.22.27905\bin[b]HostX86[/b]\x64 folder.
I had added the link C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.22.27905\bin[b]Hostx64[/b]\x64, which differs to your link by the bolded ‘HostX86 vs Hostx64’ and I got the following error ‘Compiler ‘cl.exe’ in PATH different than the one specified with -ccbin’.

I’m now going to try your version and also your other suggestions and let you know of my progress. Thank you for your support once again - its greatly appreciated.

Elton · February 1, 2020, 8:49am

No problem, happy to help. When you generate the log file with /v:diag as described in the last part, can you please zip it and attach it? Thanks.

smukachi · February 1, 2020, 8:49am

Getting there. After adding your version of the MSVC, the 'Compiler ‘cl.exe’ in PATH different than the one specified with -ccbin’error is now gone, but still having compiler problems. I have attached the /v:diag txt file for your analysis.
HelloCuda.zip (29.4 KB)

smukachi · May 6, 2020, 5:54pm

Elton, thought this might help. Ran /v:diag in command prompt without directing to file so I can get the benefits of error highlighting. The Only error I see is this:

C:\Program Files (x86)\Microsoft Visual Studio019\Enterprise\MSBuild\Microsoft\VC\v160\Microsoft.CppBuild.targets(376
,5): error MSB8013: This project doesn't contain the Configuration and Platform combination of Debug|Win32. [C:\Working
\NVIDIA\HelloCuda\HelloCuda.vcxproj]

Have windows 10, x64 and downloaded the CUDA 10.1 Toolkit from https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=10, which I believe is correct for my system specs. In VS project, I cannot pick x86, its x64 platform - interesting…

I hope this helps. I had figured out from the error message that there was a conflict with the build targets, but my skills are limited, so was stuck. I have alredy learnt a few great skills from you, thanks.

Elton · February 1, 2020, 8:49am

I’m guessing the Win32 platform configuration is missing, to resolve that add /p:Platform=x64 to the msbuild command-line.

Elton · February 1, 2020, 8:49am

Looking at your build log file, it looks like the path has both HostX86\x86 and HostX86\x64 paths, this looks suspicious to me. Hopefully explicitly specifying x64 platform resolves that. What platform were you trying to build within VS?

smukachi · February 1, 2020, 8:49am

I’m trying to build to x64. Indeed I have both:

C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.22.27905\bin\Hostx64
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.22.27905\bin\Hostx86

Which should it be?

smukachi · February 29, 2020, 1:30am

running /p:Platform=x64 results in an error:

C:\Working\NVIDIA\HelloCuda>msbuild /p:Platform=x64
Microsoft (R) Build Engine version 16.2.37902+b5aaefc9f for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Building the projects in this solution one at a time. To enable parallel build, please add the "-m" switch.
Build started 10/09/2019 23:30:09.
Project "C:\Working\NVIDIA\HelloCuda\HelloCuda.sln" on node 1 (default targets).
ValidateSolutionConfiguration:
  Building solution configuration "Debug|x64".
Project "C:\Working\NVIDIA\HelloCuda\HelloCuda.sln" (1) is building "C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj" (2) on node 1 (default targets).
InitializeBuildStatus:
  Touching "x64\Debug\HelloCuda.tlog\unsuccessfulbuild".
AddCudaCompileDeps:
Skipping target "AddCudaCompileDeps" because all output files are up-to-date with respect to the input files.
AddCudaCompilePropsDeps:
Skipping target "AddCudaCompilePropsDeps" because all output files are up-to-date with respect to the input files.
WriteCudaCompileTlogs:
Skipping target "WriteCudaCompileTlogs" because all output files are up-to-date with respect to the input files.
Project "C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj" (2) is building "C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj" (2:2) on node 1 (CudaBuildCore target(s)).
CudaBuildCore:
  Compiling CUDA source file kernel.cu...
  cmd.exe /C "C:\Users\Shephard Mukachi\AppData\Local\Temp\tmp963a284cb80949f6a27f713479e54176.cmd"
  "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe" -gencode=arch=compute_35,code=\"sm_35,compute_35\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio0
  19\Enterprise\VC\Tools\MSVC4.22.27905\bin\HostX86\x64" -x cu  -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\inc
  lude"  -G   --keep-dir x64\Debug -maxrregcount=0  --machine 64 --compile -cudart static  -g   -DWIN32 -DWIN64 -D_DEBUG -D_CONSOLE -D_MBCS -Xcompiler "/EHsc /W3 /nologo /Od /Fdx64\Debug\vc142.pdb /
  FS /Zi /RTC1 /MDd " -o x64\Debug\kernel.cu.obj "C:\Working\NVIDIA\HelloCuda\kernel.cu"

  C:\Working\NVIDIA\HelloCuda>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe" -gencode=arch=compute_35,code=\"sm_35,compute_35\" --use-local-env -ccbin "C:\Program Files (x86
  )\Microsoft Visual Studio019\Enterprise\VC\Tools\MSVC4.22.27905\bin\HostX86\x64" -x cu  -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" -I"C:\Program Files\NVIDIA GPU Comp
  uting Toolkit\CUDA\v10.1\include"  -G   --keep-dir x64\Debug -maxrregcount=0  --machine 64 --compile -cudart static  -g   -DWIN32 -DWIN64 -D_DEBUG -D_CONSOLE -D_MBCS -Xcompiler "/EHsc /W3 /nologo
  /Od /Fdx64\Debug\vc142.pdb /FS /Zi /RTC1 /MDd " -o x64\Debug\kernel.cu.obj "C:\Working\NVIDIA\HelloCuda\kernel.cu"
C:\Program Files (x86)\Microsoft Visual Studio019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations\CUDA 10.1.targets(764,9): error MSB3721: The command ""C:\Program Files\NVIDIA GPU Comput
ing Toolkit\CUDA\v10.1\bin\nvcc.exe" -gencode=arch=compute_35,code=\"sm_35,compute_35\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio019\Enterprise\VC\Tools\MSVC4.22.279
05\bin\HostX86\x64" -x cu  -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include"  -G   --keep-dir x64\Debug -maxrr
egcount=0  --machine 64 --compile -cudart static  -g   -DWIN32 -DWIN64 -D_DEBUG -D_CONSOLE -D_MBCS -Xcompiler "/EHsc /W3 /nologo /Od /Fdx64\Debug\vc142.pdb /FS /Zi /RTC1 /MDd " -o x64\Debug\kernel.c
u.obj "C:\Working\NVIDIA\HelloCuda\kernel.cu"" exited with code 1. [C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj]
Done Building Project "C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj" (CudaBuildCore target(s)) -- FAILED.

Done Building Project "C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj" (default targets) -- FAILED.

Done Building Project "C:\Working\NVIDIA\HelloCuda\HelloCuda.sln" (default targets) -- FAILED.


Build FAILED.

"C:\Working\NVIDIA\HelloCuda\HelloCuda.sln" (default target) (1) ->
"C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj" (default target) (2) ->
"C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj" (CudaBuildCore target) (2:2) ->
(CudaBuildCore target) ->
  C:\Program Files (x86)\Microsoft Visual Studio019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations\CUDA 10.1.targets(764,9): error MSB3721: The command ""C:\Program Files\NVIDIA GPU Comp
uting Toolkit\CUDA\v10.1\bin\nvcc.exe" -gencode=arch=compute_35,code=\"sm_35,compute_35\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio019\Enterprise\VC\Tools\MSVC4.22.2
7905\bin\HostX86\x64" -x cu  -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include"  -G   --keep-dir x64\Debug -max
rregcount=0  --machine 64 --compile -cudart static  -g   -DWIN32 -DWIN64 -D_DEBUG -D_CONSOLE -D_MBCS -Xcompiler "/EHsc /W3 /nologo /Od /Fdx64\Debug\vc142.pdb /FS /Zi /RTC1 /MDd " -o x64\Debug\kernel
.cu.obj "C:\Working\NVIDIA\HelloCuda\kernel.cu"" exited with code 1. [C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj]

    0 Warning(s)
    1 Error(s)

smukachi · February 1, 2020, 8:49am

I misunderstood your previous instruction of specifying a platform, now ran this:
msbuild /v:diag HelloCuda.vcxproj /p:Configuration=Debug /p:Platform=x64

Results are attached. The error is what I get in VS Output.

"C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj" (default target) (1) ->
"C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj" (CudaBuildCore target) (1:2) ->
(CudaBuildCore target) ->
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations\CUDA 10.1.targets(764,9): error MSB3721: The command ""C:\Program Files\NVIDIA GPU Comp
uting Toolkit\CUDA\v10.1\bin\nvcc.exe" -gencode=arch=compute_35,code=\"sm_35,compute_35\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.22.2
7905\bin\HostX86\x64" -x cu  -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include"  -G   --keep-dir x64\Debug -max
rregcount=0  --machine 64 --compile -cudart static  -g   -DWIN32 -DWIN64 -D_DEBUG -D_CONSOLE -D_MBCS -Xcompiler "/EHsc /W3 /nologo /Od /Fdx64\Debug\vc142.pdb /FS /Zi /RTC1 /MDd " -o x64\Debug\kernel
.cu.obj "C:\Working\NVIDIA\HelloCuda\kernel.cu"" exited with code 1. [C:\Working\NVIDIA\HelloCuda\HelloCuda.vcxproj]

I created the trivial example (HelloCuda) project using the CUDA Template. The project does not have an option to select Platform x86, I know I can add it manually, but my understading is that CUDA Toolkit 10.1 is only for x64 or did I misunderstand?

Thanks

HelloCuda.zip (66.3 KB)

Elton · February 1, 2020, 8:49am

You should only be building for x64. I realized that the error produced by nvcc.exe on the command-line may not be the same as from msbuild or VS because the build system sets the path. When you tested the nvcc.exe command-line before, did you do it from a VS developer command-prompt so there would be a cl.exe in the path?

Another thing to try, run the MSBuild command line with this and see if we get any more info:

msbuild /v:diag HelloCuda.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:CudaLogStandardErrorAsError=true

smukachi · February 1, 2020, 8:49am

I will try your instructions and will get back to you. I’ve had to reset my laptop, so re-installing the software at the moment.

smukachi · February 29, 2020, 1:30am

Hi Elton, I’m back with a clean install. Before I install Nvidia CUDA, may please confirm that the installation steps on this link are correct:

https://developer.nvidia.com/gameworksdownload#?dn=nsight-visual-studio-edition-2019-3

It might be me but there seems a bit of confusion. Steps are broadly;

Step 1: Download and install the recommended NVIDIA Display Driver

Is this the latest driver for the Display Card or a specific driver that macthes the CUDA Toolkit?
There are drivers in CUDA Toolkit, what does this mean, after this step has been done already?
I currently have Nvidia Geforce GTX 1050 ti 431.86, HDC, Studio Driver. Is this correct?

Step 2: Download and install the CUDA Toolkit 10.1 Update 2

This toolkit, as I mentioned above, has Drivers in it!
It also contains Nsight Visual Studio Edition, which is supposed to be installed in Step 3!

Step 3: Download and install Nsight Visual Studio Edition installer

Is this necessary if CUDA Toolkit already contains the installer?

Please help me clarify the installation steps. I searched around online for the past few days in my numerous attempts and cannot really say which is the proper progression. I think this might be the cause of my problems.

Thank you for your help again.

smukachi · February 1, 2020, 8:49am

Dear Elton,

Great joy! After working all night on re-installing my system and the NVIDIA CUDA package, my project now works.

My steps, which are different from my previous attempts are:

Step 1: Update NVIDIA Graphics Card to the latest update.

Step 2: Install CUDA Toolkit 10.1 using Custom Installation - then unticking the Driver Options

Step 3: Install Nsight visual studio 2019.3 which replaces the outdated Nsight visual studio it installed in Step 2

Step 4: Add the folder to MSCV to the PATH in environment variables

C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.22.27905\bin\Hostx64\x64
I can now compile from within Visual Studio and through the command line as you instructed.

Thank you a million for your support. This had to work because its part of a project I have been working on for 4 years.

Thank you,

Shep

Elton · February 1, 2020, 8:49am

That’s great news! Happy to hear that everything’s up and running again.

s3vv4 · March 10, 2020, 2:55pm

I have a similar problem, is the gist here to reinstall Windows?

Elton · March 10, 2020, 2:55pm

It should not be required to reinstall Windows to fix this. Can you provide more details about the problem you are seeing?