# TensorRT model is double size

**URL:** <https://forums.developer.nvidia.com/t/tensorrt-model-is-double-size/121075>\
**Category:** TensorRT\
**Created:** [April 25, 2020, 11:52pm UTC](https://forums.developer.nvidia.com/t/tensorrt-model-is-double-size/121075 "2020-04-25T23:52:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![DDD](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@DDD](https://forums.developer.nvidia.com/u/DDD)\
**Post date:** [April 25, 2020, 11:52pm UTC](https://forums.developer.nvidia.com/t/tensorrt-model-is-double-size/121075/1 "2020-04-25T23:52:32Z")

</div>

## Description

Hello  
I used [nvcr.io/nvidia/tensorflow](http://nvcr.io/nvidia/tensorflow) :19.09-py3  
docker run -it --rm --gpus all -v /home/dalalalmotwaa/Desktop/tf\_docker:/workspace/trt [nvcr.io/nvidia/tensorflow:19.09-py3](http://nvcr.io/nvidia/tensorflow:19.09-py3)

I have tried to optimize my custom frozen model to run on TensorRT using create\_inference\_graph()

trt\_graph = trt.create\_inference\_graph(  
input\_graph\_def=frozen\_graph,# frozen model  
outputs=output\_names,  
max\_batch\_size=128,# specify your max batch size

# max\_workspace\_size\_bytes=2\*(10\*\*9),# specify the max workspace

```
precision_mode="FP32" , is_dynamic_op=True )	

```

with gfile.FastGFile(MODEL\_PATH+“\_TensorRT\_model\_FP16\_128\_tf\_docker.pb”, ‘wb’) as f:  
f.write(trt\_graph.SerializeToString())  
print(“TensorRT model is successfully stored!”)

but the TensorRT model is slower than frozen model ,also is double size

Note : I used this repository to convert the model : [GitHub - ardianumam/Tensorflow-TensorRT: This repository is for my YT video series about optimizing a Tensorflow deep learning model using TensorRT. We demonstrate optimizing LeNet-like model and YOLOv3 model, and get 3.7x and 1.5x faster for the former and the latter, respectively, compared to the original models.](https://github.com/ardianumam/Tensorflow-TensorRT)

---- size ----  
frozen model size is 72.6 MB  
FP16 model size is 145.3 MB  
---- time ----  
TensorRT model  
4.43951940536499 ms  
1.1245734691619873 ms  
0.3470771312713623 ms  
0.244171142578125 ms  
0.24429726600646973 ms  
0.2525475025177002 ms  
frozen model  
0.256026029586792 ms  
0.2516140937805176 ms  
0.25358080863952637 ms  
0.2555859088897705 ms  
0.24196743965148926 ms  
0.22859597206115723 ms  
0.24667620658874512 ms

## Environment

**TensorRT Version** : 6.0.1  
**GPU Type** : GeForce RTX 2080 Ti  
**Nvidia Driver Version** : 418.87.00  
**CUDA Version** : 10.1.243  
**CUDNN Version** : 7.6.4  
**Operating System + Version** : Ubuntu 18.04  
**Python Version (if applicable)**: 3.6  
**TensorFlow Version (if applicable)**: 1.14.0.  
**Baremetal or Container (if container which image + tag)**: [TensorFlow Release 19.10 - NVIDIA Docs](https://docs.nvidia.com/deeplearning/frameworks/tensorflow-release-notes/rel_19.10.html#rel_19.10)

---

<div class="post-metadata">

**Author:** ![SunilJB](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/suniljb/32/14043_2.png) [@SunilJB](https://forums.developer.nvidia.com/u/SunilJB)\
**Post date:** [April 26, 2020, 6:07pm UTC](https://forums.developer.nvidia.com/t/tensorrt-model-is-double-size/121075/2 "2020-04-26T18:07:28Z")

</div>

Hi,

Can you try using latest TRT version on your system?

Also, if possible please try Yolo-\>ONNX-\> TRT approach for better performance. For any supported layer you have to create a custom plugin.  
Please refer to below sample:

> **[Sample Support Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/sample-support-guide/index.html)**
>
> This Samples Support Guide provides an overview of all the supported NVIDIA TensorRT 8.4.3 samples included on GitHub and in the product package. The TensorRT samples specifically help in areas such as recommenders, machine comprehension, character...

“trtexec” command line tool is also very useful for testing, debugging and bench-marking your model:  
[https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec](https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec)

Thanks  
Thanks

---

<div class="post-metadata">

**Author:** ![DDD](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@DDD](https://forums.developer.nvidia.com/u/DDD)\
**Post date:** [April 27, 2020, 8:04am UTC](https://forums.developer.nvidia.com/t/tensorrt-model-is-double-size/121075/3 "2020-04-27T08:04:33Z")

</div>

> [@SunilJB](#):
>
> Also, if possible please try Yolo-\>ONNX-\> TRT approach for better performance. For any supported layer you have to create a custom plugin.

Thank you very much I will try Yolo-\>ONNX-\> TRT  
What about resnet models ?

---

<div class="post-metadata">

**Author:** ![SunilJB](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/suniljb/32/14043_2.png) [@SunilJB](https://forums.developer.nvidia.com/u/SunilJB)\
**Post date:** [April 27, 2020, 8:09am UTC](https://forums.developer.nvidia.com/t/tensorrt-model-is-double-size/121075/4 "2020-04-27T08:09:02Z")

</div>

You can try the same approach for other model as well, like pb-\> ONNX-\> TRT or pt → ONNX → TRT

You can refer to TRT samples:  
[https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource](https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource)

Thanks