Visualizing tensorRT engine

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Is it possible to visualize a tensorRT engine(optimized model), so I can see the model structure, just like visualize a tensorflow model(.pb file)?

Thanks

2

Hello,

I assume you are asking for something analogous to tensorboard or netron? Unfortunately, I’m not aware a way for TRT.

3

yes, I am asking is there a tool like tensorboard or netron that I can see the TRT engine, if not, is it possible that convert TRT engine to tensorflow PB format or any other format, like ONNX, then I can use tensorboard or netron to see the optimized network?

4

Just store your TRT_graph using code below, and upload to netron browser (or use tensorboard), you will see “TRTEngineOp” there if your model is successfully optimized by TensorRT.

with gfile.FastGFile("trt_graph.pb", 'wb') as f:
        f.write(your_trt_graph.SerializeToString())
print("TRT model is stored!")
5

The netron can open onnx foramt model, but cannot open engine format model, but the onnx model i have is the original model that is not optimized, so i want to know can we output a onnx format model this is optimized by tensorrt ?

7

You can use TREx (TensorRT Engine Explorer) to visualize a TensorRT engine’s data-dependency graph (as PNG or SVG; or exported in a format for Netron). TREx has an API and can show various stats about an engine.

2 Likes
11

Quick and dirty “visualization”

trtexec --loadEngine=best.engine --dumpLayerInfo
2 Likes
12

@nzmora I tried the TRTExplorer, but it doesn’t seem to work on Jetson. Am I mistaken? Or is there a plan to offer this on jetson? Or perhaps as an online tool that we can use in a browser on Jetson?

13

Hi @duane.nielsen.rocks,
I have not tried running TREx on the Jetson platform itself, only on Ubuntu machines so it’s possible that something is going wrong (you may have forgotten to explain the specifics of what “doesn’t work on Jetson”).
However, the inputs to TREx are just JSON files so you can grab these files and use them as inputs to TREx on a workstation - you don’t need a GPU to run TREx.
BTW TREx is a library that you can use to analyze an engine’s characteristics and doesn’t require a display. You can also run it remotely via a notebook server to view rendered graphs and diagrams in your browser.

14

Right… I was suggesting perhaps you could host the tool online? I get that I can host it, but it’s now 2022… you know :)