How to set context inputDims when there are two inputs to the network and two outputs

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

In keras my input looks like this:
image
Its a list of two tensors

And my outputs:
image

I’m working with the “simpleOnnx” example and I wonder how to set the input tensor and output tensors:
(Based my conde on https://developer.nvidia.com/blog/speed-up-inference-tensorrt/)

For one inputTensor I used this code:

    Dims dims_i{engine->getBindingDimensions(0)};
    Dims3 inputDims{batchSize, dims_i.d[1], dims_i.d[2]};
    context->setBindingDimensions(0, inputDims);
    launchInference(context.get(), stream, inputTensor, outputTensor, bindings, batchSize);

Any suggestions? or maybe an example for multiple inputs and outputs?

Environment

TensorRT Version : 7.0.0-1
GPU Type : RTX
Nvidia Driver Version :
CUDA Version : 11.0
CUDNN Version :
Operating System + Version : Ubuntu 18.04
Python Version (if applicable) : 3.6
TensorFlow Version (if applicable) : 2.3
PyTorch Version (if applicable) :
Baremetal or Container (if container which image + tag) :

Update:
I’ve tried to change
Dims3 inputDims{batchSize, dims_i.d[1], dims_i.d[2]}
to
Dims4 inputDims{batchSize, 2, dims_i.d[1], dims_i.d[2]}

But I’m getting:
size outputTensor : 4
size inputTensor: 495
0 0 0 0 ERROR: Parameter check failed at: engine.cpp::resolveSlots::1092, condition: allInputDimensionsSpecified(routine)

2

Hi @shir

Could you please check if you have set the input size for the preprocessor during inference using setBindingDimensions?
Please refer below sample:

Thanks!

3

Hi,
Thanks for you answer.
I have set the bindingDimensions like this:

Dims3 inputDims{1, 55, 3};
context->setBindingDimensions(0, inputDims);

I’m not sure that it’s the right way cause as I said above, I have two input tensors with the size of (55,3)
But I’m still getting this error:
ERROR: Parameter check failed at: engine.cpp::resolveSlots::1092, condition: allInputDimensionsSpecified(routine

Any suggestions on how to set the “binding dimensions”?
Any example on how to use with multiple outputs and inputs?

Update:
Fixed it by setting the binding dimensions for both of my inputs, like this:
context->setBindingDimensions(0, inputDims);
context->setBindingDimensions(1, inputDims);

But I’ve tried to insert a tensor of ones in the size of both my inputs, both in Keras and in tensorRT but didn’t got the same results? any idea why it happens ?

4

Hi @shir,
Apologies for delayed response
Are you still facing the issue?
Can you please share the model with us, as the error shows that there might be some issue with your model.
Thanks!