How to use Nsight compute "source counter" to capture the CUDA kernel code in PyTorch?

weipenghui_666 · September 22, 2023, 8:21am

Hi, I want to use the “source counter” of nsight compute to see the execution of CUDA kernel and SASS instructions. I have rebuild pytorch with addition flags:

USE_CUDA=1 DEBUG_CUDA=1 python setup.py develop ===> Success build Pytorch

In CMakeLists.txt of Pytorch, DEBUG_CUDA enables the “–lineinfo”. But after I try again to profile a torch.mm or torch.softmax, it still can not see any CUDA C kernel correlation SASS in “source counter”. I wonder to know where is the problem or it is impossible to perform this ?

github.com

pytorch/pytorch/blob/d7c05bb2e8de24386664c01e887357ff50a09842/CMakeLists.txt#L742


      
            USE_FLASH_ATTENTION
            "Whether to build the flash_attention kernel for scaled dot product attention" ON
            "USE_CUDA AND NOT ROCM AND NOT MSVC AND NOT CMAKE_CUDA_COMPILER_VERSION VERSION_LESS 11.6" OFF)
          
          # Flash Attention2 will error while building for sm52 while Mem Eff Attention won't
          cmake_dependent_option(
            USE_MEM_EFF_ATTENTION
            "Enable memory-efficient attention for scaled dot product attention" ON
            "USE_CUDA AND NOT ROCM AND NOT CMAKE_CUDA_COMPILER_VERSION VERSION_LESS 11.6" OFF)
          
          if(DEBUG_CUDA)
            string(APPEND CMAKE_CUDA_FLAGS_DEBUG " -lineinfo")
            string(APPEND CMAKE_CUDA_FLAGS_RELWITHDEBINFO " -lineinfo")
            # CUDA-12.1 crashes when trying to compile with --source-in-ptx
            # See https://github.com/pytorch/pytorch/issues/102372#issuecomment-1572526893
            if(CMAKE_CUDA_COMPILER_VERSION VERSION_LESS 12.1)
              string(APPEND CMAKE_CUDA_FLAGS_DEBUG " --source-in-ptx")
              string(APPEND CMAKE_CUDA_FLAGS_RELWITHDEBINFO " --source-in-ptx")
            endif()
          endif(DEBUG_CUDA)

I also try to add “-lineinfo” in CUDA_NVCC_FLAGS, but still not work.

github.com

pytorch/pytorch/blob/d7c05bb2e8de24386664c01e887357ff50a09842/cmake/public/cuda.cmake#L388


      
              string(APPEND CMAKE_CUDA_FLAGS_RELWITHDEBINFO " -Xcompiler /MD")
            endif()
            if(CUDA_NVCC_FLAGS MATCHES "Zi")
              list(APPEND CUDA_NVCC_FLAGS "-Xcompiler" "-FS")
            endif()
          elseif(CUDA_DEVICE_DEBUG)
            list(APPEND CUDA_NVCC_FLAGS "-g" "-G")  # -G enables device code debugging symbols
          endif()
          
          # Set expt-relaxed-constexpr to suppress Eigen warnings
          list(APPEND CUDA_NVCC_FLAGS "--expt-relaxed-constexpr")
          
          # Set expt-extended-lambda to support lambda on device
          list(APPEND CUDA_NVCC_FLAGS "--expt-extended-lambda")
          
          foreach(FLAG ${CUDA_NVCC_FLAGS})
            string(FIND "${FLAG}" " " flag_space_position)
            if(NOT flag_space_position EQUAL -1)
              message(FATAL_ERROR "Found spaces in CUDA_NVCC_FLAGS entry '${FLAG}'")
            endif()
            string(APPEND CMAKE_CUDA_FLAGS " ${FLAG}")

Resolved, I update to newest nsight compute, it works

jmarusarz · September 26, 2023, 8:03pm

Thanks for letting us know that the update helped.

jmarusarz · November 15, 2023, 4:13pm