Sample code of feeding image into TensorTR inference engine

Here is an example of performing the inference with engine build with TensorRT.
Suppose I’ll build an engine from VGG16. Is there some code examples of how to pass-in some image file (jpg, png) into the TensorTR inference engine? And how to get the inference result?

Thanks in advance!


You can use TRT to optimize any model file and generate the TRT engine as long as operations in network are supported. Otherwise you might have to create a custom plugin for that particular operation.

Please refer below link to generate TRT engine file:

Also, please refer to below sample for end-end workflow:


I’ve read the documents, and I’ve accomplished steps of building an engine following instruction in the second of you link.

My question is there a sample code of using built engine: how to provide the engine with input and receive an output?

There are few words in the doc about it (the link in my first message) but there is no complete example.


You can refer to below python sample example:

Here inference part is handled in “” and data processing (pre-post) can be found in “” file. Pre-processing of the input data will depend on the application & model you are creating.


@SunilJB Thank you very much! The code examples are exactly what I was looking for.

I also wanted to execute the Yolo sample but it returns an error.

Traceback (most recent call last):
  File "", line 811, in <module>
  File "", line 804, in main
  File "/usr/local/lib/python2.7/dist-packages/onnx/", line 91, in check_model
onnx.onnx_cpp2py_export.checker.ValidationError: Node (086_upsample) has input size 2 not in range [min=3, max=4].

That is strange 'cause everything is very much “standard”, I mean it uses the same as supposed yolo’s config from the yolo git repo, I haven’t changed the script, I executed it on Jetson nano with JetPack 4.3, etc.
Have I missed something?

I put the whole output of here

Any recommendations are appreciated!


Based on the error log, it seems that the “Resize” operation only has two input params, as per the ONNX operation document “Resize” operation should have input size of (3-4). [X : T1, roi : T2, scales : tensor(float), sizes (optional) : tensor(int64)]

Current model just have two input params:
name: “086_upsample” op_type: “Resize” attribute { name: “mode” s: “nearest” type: STRING }

Please refer below link for more details:


@SunilJB thank you a lot for your help!
Based on your examples I managed to create a simple code which processes data via generated TensorRT engine.
I put the code in case if someone will need it
It’s much simpler than the yolo example since it’s based on just mnist )

In order to try it first generate the LeNet5(MNIST) model following this guide

Couldn’t fix the yolov3 sample though, since I don’t know well ONNX.
Please let me know if someone got it working


Could you please elaborate more on the error you are getting in yolov3 samples? Is it the same resize operation issue?
If possible,please share the error log.


Hi! yes, this is the same resize operation issue, still cannot fix it.
I put the whole output of here Did you mean this log?
Please let me know if some additional info will be helpful!


As mentioned in earlier post, current “Resize” operation implementation in your model is not as per the ONNX supported operation.
You have to either update your model file or write a custom plugin to handle the customized “Resize” operation.