file size of .pb increased after optimization

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

hello,

i am running the https://developer.download.nvidia.com/devblogs/tftrt_sample.tar.xz in nvidia/tensorflow:19.01-p3 container with Tesla V100 16GB.

when running the provided script for optimization, the file sizes of the respective graphs seem to increse.

root@2979b9610c8a:/workspace/tftrt_sample# ls -l
total 987852
-rwxrwxr-x 1 root root       294 Mar 24  2018 README
-rw-rw-r-- 1 root root     61098 Mar 24  2018 grace_hopper.jpg
-rwxrwxr-x 1 root root     31587 Mar 24  2018 labellist.json
-rw-rw-r-- 1 root root 218641975 Mar  1 10:16 resnetV150_TRTFP16.pb
-rw-rw-r-- 1 root root 280046959 Mar  1 10:15 resnetV150_TRTFP32.pb
-rw-rw-r-- 1 root root 205066441 Mar  1 10:19 resnetV150_TRTINT8.pb
-rw-rw-r-- 1 root root 205062671 Mar  1 10:16 resnetV150_TRTINT8Calib.pb
-rw-r--r-- 1 root root 102591940 Mar 24  2018 resnetV150_frozen.pb
-rwxrwxr-x 1 root root       542 Mar 24  2018 run_all.sh
-rw-r--r-- 1 root root     12503 Mar 24  2018 tftrt_sample.py

the trt graph with fp32 is alsmost 3 times the size as the unoptimized.
apart from that, optimization seems to work fine with inference times of:
~7ms unoptimized, ~5ms FP32, ~3ms FP16, ~3ms INT8

2

it should look like this:

root@2979b9610c8a:/workspace/copy/tftrt# ls -l
total 381600
-rwxrwxr-x 1 1000 1000       294 Mar 24  2018 README
-rw-rw-r-- 1 1000 1000     61098 Mar 24  2018 grace_hopper.jpg
-rwxrwxr-x 1 1000 1000     31587 Mar 24  2018 labellist.json
-rw-rw-r-- 1 1000 1000  51170247 Mar 24  2018 resnetV150_TRTFP16.pb
-rw-rw-r-- 1 1000 1000 102203679 Mar 24  2018 resnetV150_TRTFP32.pb
-rw-rw-r-- 1 1000 1000  32085909 Mar 24  2018 resnetV150_TRTINT8.pb
-rw-rw-r-- 1 1000 1000 102559003 Mar 24  2018 resnetV150_TRTINT8Calib.pb
-rw-r--r-- 1 1000 1000 102591940 Mar 24  2018 resnetV150_frozen.pb
-rwxrwxr-x 1 1000 1000       542 Mar 24  2018 run_all.sh
-rw-r--r-- 1 1000 1000     12503 Mar 24  2018 tftrt_sample.py
3

Yes. the size is expected. one copy for TRT, one copy for TF Native execution.

4

Thanks for clarrifying.

Is there any way how I can save and use only the TRT optimizied version, so I can get the same file sizes and infere over them as in: