NVIDIA Developer Forums

Jetson AGX Xavier: slow inference using CUDA and PyTorch

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier

AastaLLL April 13, 2021, 12:22pm 5

Hi,

You will need to use the Nsight Compute integrated in the JetPack.
Please install the CUDA package for host and find it in /opt/nvidia/nsight-compute/2019.5.0.

Thanks.

Topic		Replies	Views	Activity
Slow inference using CUDA and PyTorch on Jetson AGX CUDA Programming and Performance cuda , jetson-inference , pytorch	4	669	November 2, 2021
Clearing cache in xavier agx improves speed of copy from cpu to gpu TensorRT cuda , jetson-inference	3	776	April 11, 2022
Transfer rate from GPU to CPU with pytorch on Xavier NX Jetson Xavier NX jetson-inference	2	706	March 9, 2022
Inference is so slow with torch1.6 Jetson Xavier NX nvbugs , pytorch	12	3565	October 23, 2020
Jetson AGX Xavier shows unstable inference time Jetson AGX Xavier tensorrt , jetson-inference	6	717	October 18, 2021
Pycuda runs super slow on Jetson Xavier NX compared to running on CPU Jetson Xavier NX pycuda	8	1869	October 18, 2021
Why is torch.tensor.to('cuda') so slow? Jetson AGX Orin pytorch	5	106	December 9, 2024
Jetson xavier nx is slow Jetson Xavier NX jetson-inference	4	61	February 12, 2025
Simple CUDA example 4x slower on Xavier AGX GPU than CPU Jetson AGX Xavier cuda	3	516	October 18, 2021
Yolov5 slow inference on Jetson Xavier NX16 Jetson Xavier NX ai	10	1637	October 26, 2022