If I drop the number of #s to around 500 (and do the same in main.cu so deleted functions are not called), I encounter the following (different) error:
There may be things that need to be ironed out with the nvcc toolchain and large projects. This could be a case, and it’s usually useful to file a bug.
Thanks for the very fast response as usual, txbob.
Before I upload my project I would like to exhaust all options available, as the project contains around 10k files and takes a very long time to build (~10 hrs with VS build command and ~4 hrs with Incredibuild).
The MSB6001 error makes me think whether this error could be related to the exceedingly long nvcc -dlink command, as the file path of every single .cu.obj were appended.
-rdc and device link go hand in hand. I’m not sure what you mean by -dlink without -rdc
It might be that the error is related to the long command line. But you said when you shortened it, you got a different error. There might be multiple issues.
Windows (IMHO) tends to have less flexibility for trying to narrow these things down. If you can easily try a test build under linux, that might shed some light on it. It should be possible to shorten the command line length considerably under linux, I would think.
FWIW the previous link I gave required approximately a 24-hour build time (under linux, and not using make -j)
After manually executing the linking in cmd I am now pretty sure the error’s due to the long command line. The second error (#~=500) is simply a “The command line is too long” in cmd. The first error (#~=4000) becomes “The system cannot execute the specific program.”, and the command is cut off after some number of characters.
I’ve also considerably shorten the file names and am launching the command directly in x64\Release. The file names in the command now looks like … 123.o 124.o 125.o … etc. After doing this I still can only cram ~1000 files into the command line, way below the required 4000.
Are you sure this is not simply down to the Windows system limit on the length of command lines? The following write-up states that there is a 8191 character limit: [url]https://support.microsoft.com/en-us/kb/830473[/url]. I would suggest changing the mechanics of the build. You could add groups of, say, 100 files at a time to a static library, then link your app against that library.
[Later:] I remembered that Microsoft’s C/C++ compiler, being aware of the limitation on command lines, actually supports a “linker response file” as an alternative way of passing the information to the linker. However, this is apparently not supported by Visual Studio: [url]Microsoft Docs - Developer tools, technical documentation and coding examples.
I do not have much insight into integrating third party tools with Visual Studio (last looked at this at the turn of the century), but it looks like these Microsoft imposed limitations may not be trivial for NVIDIA to work around. I would suggest filing an enhancement request with NVIDIA, using the bug reporting form linked from the registered developer website. It will help to prefix the synopsis with “RFE:” to mark it as an enhancement request rather than a functional bug.
njuffa, this does indeed look like an issue with Windows’s limit on command line string length.
I have restructured my code and cut the size down to 1/8 of original and put the 4096 functions in 64 separate files instead of 4096 ones. The full rebuild time with Incredibuild is reduced from 4 hrs to 27 min, and my program runs well.
Thank you again for the detailed response, txbob and njuffa.
