Running TensorRT inference in a Python Thread


class Stream:
    def __init__(self,camera, detector): = cv2.VideoCapture(camera)
        self.grabbed, self.frame =
        self.stopped = False
        self.detector = detector

    def start_reading(self):
        Starts a new thread to read frames
        Thread(target=self.read_frames, args=()).start()
        return self

    def read_frames(self):
        Method called in a thread to continually read frames from `self.cap`.
        This way, a frame is always ready to be read. Frames are not queued;
        if a frame is not read before `read_frames()` reads a new frame, previous
        frame is overwritten.
        while not self.stopped:
            if not self.grabbed:
                self.grabbed, self.frame =
    def stop(self):
        self.stopped = True

self.detector.infer(self.frame) is failing when run inside the class.
I get the following outputs:

[12/30/2022-19:13:05] [TRT] [E] 1: [reformat.cpp::executeCutensor::384] Error Code 1: CuTensor (Internal cuTensor permutate execute failed)
[12/30/2022-19:13:05] [TRT] [E] 1: [checkMacros.cpp::catchCudaError::278] Error Code 1: Cuda Runtime (invalid resource handle)

However, when I run inference with the TRT detector outside of the class on Stream.frame in main(), it works fine.


TensorRT Version: 8.4.1
GPU Type: Jetson Orin AGX
Nvidia Driver Version:
CUDA Version: 11.4
CUDNN Version:
Operating System + Version: Jetpack 5.0.2
Python Version (if applicable): 3.8
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):


This looks like a Jetson issue. Please refer to the below samples in case useful.

For any further assistance, we will move this post to to Jetson related forum.