Where is the create_inference_graph function in TensorRT-5

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi NVIDIA,

I’m using TensorRT-5.0.2.6GA and Tensorflow-1.12.

When I try to follow the sample ‘tr-trt’, the function create_inference_graph is missing. Can I use this function in TensorRT5?

I get sample from this page: https://devblogs.nvidia.com/tensorrt-integration-speeds-tensorflow-inference/

Here is the log:

Traceback (most recent call last):
File “tftrt_sample.py”, line 291, in
timings,comp,valfp32,mdstats=timeGraph(getFP32(f.batch_size,wsize),f.batch_size,f.num_loops,
File “tftrt_sample.py”, line 111, in getFP32
trt_graph = trt.create_inference_graph(getResnet50(), [ “resnet_v1_50/predictions/Reshape_1”],
AttributeError: module ‘tensorrt’ has no attribute ‘create_inference_graph’

Best Regards
Matthias

2

Hello,

try

from tensorflow.contrib import tensorrt as trt
1 Like
3

Hello,

It seems like that tensorflow.contrib.tensorrt only support TensorRT4.

I’m now using Tensorflow-1.12, which is already the last build of Tensorflow.

**** Failed to initialize TensorRT. This is either because the TensorRT installation path is not in LD_LIBRARY_PATH, or because you do not have it installed. If not installed, please go to https://developer.nvidia.com/tensorrt to download and install TensorRT ****
Traceback (most recent call last):
  File "tftrt_sample.py", line 25, in <module>
    from tensorflow.contrib import tensorrt as trt
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/tensorrt/__init__.py", line 34, in <module>
    raise e
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/tensorrt/__init__.py", line 25, in <module>
    from tensorflow.contrib.tensorrt.python import *
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/tensorrt/python/__init__.py", line 22, in <module>
    from tensorflow.contrib.tensorrt.python.ops import trt_engine_op
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/tensorrt/python/ops/trt_engine_op.py", line 32, in <module>
    resource_loader.get_path_to_datafile("_trt_engine_op.so"))
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/util/loader.py", line 56, in load_op_library
    ret = load_library.load_op_library(path)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/load_library.py", line 60, in load_op_library
    lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: libnvinfer.so.4: cannot open shared object file: No such file or directory
4

Hello,

the errors seem to indicate you don’t have TRT properly installed. Please double check your TensorRT installation.

5

Hello,

I try to build Tensorflow myself and the problem was solved. It seems that the releases version of Tensorflow 1.12 only support TensorRT4.

Thanks!

6

I have the same problem. I have to use TensorRT 5 for the dilated convolutions.

Did you install Tensorflow from source from here Kaynaktan oluştur  |  TensorFlow ?

Thanks

7

Yes, it works well with TRT-5.

8

This fixed my issue, thank you.