Nsight Compute with Pytorch

[mustafakhawaja93]

HI,

The nsys-rep file generated from the PyTorch works fine the nsight systems. How do I generate a file from a Pytorch code that can be loaded in Nsight Compute.

Kind Regards,
Khawaja

[veraj]

Hi, @mustafakhawaja93

You can use Nsight Compute CLI to generate a report 4. Nsight Compute CLI — NsightCompute 12.6 documentation and then load in Nsight Compute GUI.

[mustafakhawaja93]

Thanks.
I was testing a simple linear_layer code. In the CUDA version, I define the threads, Blocks and Grids and also limit the number of Kernels to 1.
But I when I write the same code in Pytorch, the number of CUDA Kernels executed at background is 3 (profiling gives me this).

import torch
import torch.cuda.profiler as profiler
import time

# Initialize tensors
host_weights = torch.tensor([0.80, 0.87, 0.16, 0.96, 0.89, 0.87, 0.31, 0.08, 0.09, 0.69, 0.03, 0.42], dtype=torch.float32)
host_inputs = torch.tensor([0.75, 0.98, 0.74, 0.28], dtype=torch.float32)
host_biases = torch.tensor([0.68, 0.83, 0.01], dtype=torch.float32)

def kernel_execution(weights, inputs, biases):
    torch.cuda.nvtx.range_push("Kernel execution")
    z = torch.matmul(weights, inputs) + biases
    activations = torch.sigmoid(z)
    torch.cuda.nvtx.range_pop()
    # print("Output Z:")
    # print(z)
    return activations

with torch.autograd.profiler.emit_nvtx():
    profiler.start()

    # Memcpy HtoD
    torch.cuda.nvtx.range_push("Memcpy HtoD")
    weights = host_weights.cuda().reshape(3, 4)
    inputs = host_inputs.cuda().reshape(4, 1)
    biases = host_biases.cuda().reshape(3, 1)
    torch.cuda.nvtx.range_pop()

    # Kernel execution
    activations = kernel_execution(weights, inputs, biases)

    # Memcpy DtoH
    torch.cuda.nvtx.range_push("Memcpy DtoH")
    host_activations = activations.cpu()
    torch.cuda.nvtx.range_pop()

    profiler.stop()

print("Activations on z:")
print(host_activations)

Is there a way I can limit the kernels and thread/block/grid size in Pytorch?

[veraj]

Can you please check this problem in “CUDA Programming” forum to get better support ?
Thanks !