Have I successully generated a Stand-Alone TensorRT plan?

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi

I would like to know if I have successfully converted a TRT_graph node to a Standalone TensorRT Plan.

I have used the git provided by NVIDIA: TensorFlow/TensorRT Models on Jetson to convert a model (Inception V1) from TF to TF-TRT.

If yes: where can I find an example on how to execute that .plan file?

If no: Do these models can execute as a .plan file: Inception V1, Inception V2, Inception V3, Inception V4?

This is the output:

Exclude Node: Placeholder, input
Node: TRTEngineOp, TRTEngineOp_0
Exclude Node: Softmax, scores
3

Hi,

The GitHub demonstrates how to convert the model into TF-TRT, not the standalone TensorRT plan.
As a result, these model need to be executed with TensorFlow interface:

For a pure TensorRT sample, here are some detection sample for your reference:

/opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_SSD/
/usr/src/tensorrt/samples/sampleUffSSD/

Thanks.

4

I have used the code provided in NVIDIA documentation: https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#tensorrt-plan

I want to know if I have successfully converted the classification model (Inception V1) into a .plan file?
I do not understand the info provided:

The script will display which nodes were excluded 
for the engine. If there are any nodes listed besides
the input placeholders, TensorRT engine, and output 
identity nodes, your engine does not include the entire model.

image
image957×188 19.5 KB

Also I want to know how to use the .plan file.

How do those online NVIDIA courses work? Will they give me code as an example?

  • OPTIMIZATION AND DEPLOYMENT OF TENSORFLOW MODELS WITH TENSORRT (30USD + TAX)
    Courses – NVIDIA

  • FUNDAMENTALS OF ACCELERATED COMPUTING WITH CUDA PYTHON (90USD + TAX)
    Courses – NVIDIA

5

Hi,

Sorry for the late update.
You can run the .plan file with our trtexec directly.

/usr/src/tensorrt/bin/trtexec --loadEngine=xxxx.plan

You can also refer to its source for the usage:

/usr/src/tensorrt/samples/trtexec

Thanks.

1 Like
6

Hi

I used TensorFlow-TRT to optimize a classification model. Then I applied some code (I found it in Nvidia’s documentation) to convert that graph to a .plan file. I just do not know if I have succeeded.

image