Solved partially.
My problem was lack of clarity on the tensor shapes for both input and output layers. In fact, it took me some time to reply because all my models where rendered useless as a result of an nvidia moderator recommending to install tensorRT 8.4 EA to solve a problem to get the tensorRT quick start running Error (Could not find any implementation for node ArgMax_260.) - #6 by david9xqqb, which is impossible because of compilation errors and other unsolved issues.
I was able to get a result from the C++ model after ignoring all Nvidia documentation. On the input layer I was able to get a piece of C++ code running from here (Thank you Cyrus), which gave me clarity on the input layer from OpenCV since the order of the color layers IS important, and created an algorithm that tested multiple tensor shapes for the output.
The output shape is a vector of probabilities for each class, arranged in a matrix to correspond to each pixel in the frame.
However, the result is quite imperfect, and identical to the Deepstream result on my other post Custom TAO unet model classifying only two classes on Deepstream!
This is the output from the C++ inference:
This is the output of the same model in Deepstream:
Both are not useful and far from the inference under tao inference:
As you can see on the top right corner of the TAO inference, there is an erroneous classification, which is the third class in both the C++ and Deepstream results