I have running large language model (LLM) code that I’d like to speed up via CUDA. I understand that I’ll need to recompile the LLM interface.
My 64-bit Windows 10 laptop has an NVIDIA GeForce RTX 3070 that I’d like to recruit.
I have tried to follow instructions in the CUDA Installation Guide for Microsoft Windows.
I have updated the 3070 driver to version 31.0.15.5222, and I have installed…
- CUDA Toolkit 12.4 Update 1, for 64-bit Windows 10
- MinGW-64 gcc, g++ 6.3.0 (no gdb)
- cmake 3.29.3
I have set the PATH environment varible to include gcc, g++, and cmake locations—which I’ve expected to satisfy this requirement in instructions: A supported version of Linux with a gcc compiler and toolchain. (I.e., I haven’t installed Linux, and I’m not sure why this is mentioned in the CUDA installation guide for Windows.) I’m not sure Visual Studio, required below, needs these external programs, but it could be good to test them out. I’m hoping to call just make or cmake, per LLM interface instructions.
Solely (?) to verify (Guide section 2.6) the above via cuda-samples, I’ve installed…
- Visual Studio 2022 Version 17.9.6, with Visual C++ 2022
When I try to open the full project Samples_VS2022.sln in Visual Studio, I get a few dozen errors like this.
error : The imported project "C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\BuildCustomizations\CUDA 12.4.props" was not found. Confirm that the expression in the Import declaration "C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\\BuildCustomizations\CUDA 12.4.props" is correct, and that the file exists on disk. C:\Users\bobsc\source\repos\cuda-samples\Samples\0_Introduction\asyncAPI\asyncAPI_vs2022.vcxproj
How should I proceed?