How to access NvInfer tesor-meta in python?


I need to use the tensor output of nvinfer plugin in python code (preferably as np.ndarray).
based on deepstream_infer_tensor_meta_test.cpp’s pgie_pad_buffer_probe function, I wrote a python function that will get the tensor-meta memory address. Here is the function:

import pyds
import pycuda.driver as cuda
import ctypes

ctypes.pythonapi.PyCapsule_GetPointer.restype = ctypes.c_void_p
ctypes.pythonapi.PyCapsule_GetPointer.argtypes = [ctypes.py_object, ctypes.c_char_p]

def extract_tensor_meta(batch_meta: pyds.NvDsBatchMeta):
    l_frame = batch_meta.frame_meta_list
    while l_frame is not None:
            frame_meta = pyds.NvDsFrameMeta.cast(
        except StopIteration:
        l_user = frame_meta.frame_user_meta_list
        while l_user is not None:
                user_meta = pyds.NvDsUserMeta.cast(
            except StopIteration:

            meta_type = user_meta.base_meta.meta_type
            if meta_type == pyds.NVDSINFER_TENSOR_OUTPUT_META:
                meta = pyds.NvDsInferTensorMeta.cast(user_meta.user_meta_data)
                for i in range(meta.num_output_layers):
                    mem_add = ctypes.pythonapi.PyCapsule_GetPointer(
                        meta.out_buf_ptrs_host, None)
                    cuda.memcpy_dtoh(mem_add + i, 0)

                l_user =
            except StopIteration:
            l_frame =
        except StopIteration:

But the cuda.memcpy line throws

TypeError: a bytes-like object is required, not 'int'

Prior to converting the meta.out_buf_ptrs_host to memory address I simply tried

cuda.memcpy_dtoh(meta.out_buf_ptrs_host[i], meta.out_buf_ptrs_dev[i])

But I encountered:

TypeError: 'PyCapsule' object is not subscriptable

I also tried to convert the memory address into a python object by dereferencing the memory address with no success.

My setup:
Hardware Platform: GPU

DeepStream Version: 5.1
TensorRT Version:
NVIDIA GPU Driver Version: 460.73.01

Reference for pycuda.driver.memcpy: Device Interface - PyCUDA 2021.1 documentation

deepstream_infer_tensor_meta_test.cpp address:


Could you please help me convert the nvinfer tensor output into a python object?
Thanks a lot!

Okay, based on this thread’s solution, I managed to convert the tensor output to numpy array.

    frame_outputs = []
    for i in range(num_output_layers):
        layer = pyds.get_nvds_LayerInfo(meta, i)
        # Convert NvDsInferLayerInfo buffer to numpy array
        ptr = ctypes.cast(pyds.get_ptr(layer.buffer), ctypes.POINTER(ctypes.c_float))
        v = np.ctypeslib.as_array(ptr, shape=output_shapes[i])

Glad to know issue resolved.