How to cast the buffer (tensorRT inference result) of a network with 3 dimensional output

[elight1]

Description

1. I have a pytorch neural network which receives an image and produces a multi-dimensional output (e.g. size 30x15x15)

2. I managed to do the inference in TensorRT. I used the sampleOnnxMNIST sample on github.

3. What I can’t do is to extract the network outputs from the buffer:

float* output = static_cast<float*>(buffers.getHostBuffer(mParams.outputTensorNames[0]));

The pointer output points to 30 values which I loop, but if I go outside the 30 bound, I get random numbers.

    for (int i = 0; i < 30; i++)
    {
        std::cout << output[i] << "\n";
    }

Environment

TensorRT Version: 8.4.1.5
GPU Type: Nvidia
Nvidia Driver Version:
CUDA Version: 11.5
CUDNN Version:
Operating System + Version: Ubuntu 20.04
Python Version (if applicable): 3.7
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.10.1
Baremetal or Container (if container which image + tag):

[NVES]

Hi,
Can you try running your model with trtexec command, and share the “”–verbose"" log in case if the issue persist

TensorRT/samples/trtexec at master · NVIDIA/TensorRT

You can refer below link for all the supported operators list, in case any operator is not supported you need to create a custom plugin to support that operation

onnx/onnx-tensorrt/blob/main/docs/operators.md

Also, request you to share your model and script if not shared already so that we can help you better.

Meanwhile, for some common errors and queries please refer to below link:

Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

Thanks!