#line 1 "nonexist_file"
The output of
nvcc -M test.cu contains “nonexist_file”. gcc will not output this.
This issue causes C++ extensions that use PyTorch to always rebuild, because PyTorch uses this
#line preprocessor directive (e.g. pytorch/THGenerateByteType.h at master · pytorch/pytorch · GitHub). And PyTorch relies on ninja to determine if rebuilding is needed. But ninja cannot find the dependent file, so it decided to rebuild every time. This makes our program slow because
torch.utils.cpp_extension.load function will trigger a rebuild on every invocation.