Can't convert model.pb by TF-TRT

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello.

OS : Ubuntu18.04
python : 3.6.8
tensorflow-gpu : 1.13.1
GPU : TITAN RTX * 2

I built model.pb file by tf.saved_model.builder.SavedModelBuilder.
And I tried to convert this pb-file by the following command.

graph = tf.Graph()
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
config.allow_soft_placement = True

with graph.as_default():
    with tf.Session(config=config) as sess:
        # Create a TensorRT inference graph from a SavedModel:
        trt_graph = trt.create_inference_graph(
            input_graph_def=None,
            outputs=['outputs'],
            input_saved_model_dir='saved_model/1',
            output_saved_model_dir='saved_model/1_trt',
            input_saved_model_tags=['serve'],
            max_batch_size=1,
            max_workspace_size_bytes=100,
            minimum_segment_size=3,
            precision_mode='fp16',
            is_dynamic_op=True,
            maximum_cached_engines=1,
            cached_engine_batch_sizes=None,
            use_calibration=True,
            session_config=config)

But I couldn’t convert and received the following message.

W tensorflow/contrib/tensorrt/segment/segment.cc:683] Segment 0 has multiple devices attached: /device:CPU:0, /device:GPU:0, /job:localhost/replica:0/task:0/device:GPU:0,  choosing /device:CPU:0
W tensorflow/contrib/tensorrt/segment/segment.cc:683] Segment 1 has multiple devices attached: /device:CPU:0, /device:GPU:0,  choosing /device:CPU:0
W tensorflow/contrib/tensorrt/segment/segment.cc:683] Segment 2 has multiple devices attached: /device:CPU:0, /device:GPU:0,  choosing /device:CPU:0
W tensorflow/contrib/tensorrt/segment/segment.cc:683] Segment 3 has multiple devices attached: /device:CPU:0, /device:GPU:0,  choosing /device:CPU:0
W tensorflow/contrib/tensorrt/segment/segment.cc:683] Segment 4 has multiple devices attached: /device:CPU:0, /device:GPU:0,  choosing /device:CPU:0
W tensorflow/contrib/tensorrt/segment/segment.cc:683] Segment 10 has multiple devices attached: /device:CPU:0, /device:GPU:0,  choosing /device:CPU:0
W tensorflow/contrib/tensorrt/segment/segment.cc:683] Segment 11 has multiple devices attached: /device:CPU:0, /device:GPU:0,  choosing /device:CPU:0
W tensorflow/contrib/tensorrt/segment/segment.cc:683] Segment 25 has multiple devices attached: /device:CPU:0, /device:GPU:0,  choosing /device:CPU:0
I tensorflow/contrib/tensorrt/convert/convert_graph.cc:913] Number of TensorRT candidate segments: 26
W tensorflow/contrib/tensorrt/convert/convert_graph.cc:454] Detected multiple(2) devices for the segment. Picking first one to continue but this shouldn't have happened
W tensorflow/contrib/tensorrt/convert/convert_graph.cc:454] Detected multiple(2) devices for the segment. Picking first one to continue but this shouldn't have happened
W tensorflow/contrib/tensorrt/convert/convert_graph.cc:454] Detected multiple(2) devices for the segment. Picking first one to continue but this shouldn't have happened
W tensorflow/contrib/tensorrt/convert/convert_graph.cc:454] Detected multiple(2) devices for the segment. Picking first one to continue but this shouldn't have happened
W tensorflow/contrib/tensorrt/convert/convert_graph.cc:454] Detected multiple(2) devices for the segment. Picking first one to continue but this shouldn't have happened
W tensorflow/contrib/tensorrt/convert/convert_graph.cc:454] Detected multiple(2) devices for the segment. Picking first one to continue but this shouldn't have happened
W tensorflow/contrib/tensorrt/convert/convert_graph.cc:454] Detected multiple(2) devices for the segment. Picking first one to continue but this shouldn't have happened
W tensorflow/contrib/tensorrt/convert/convert_graph.cc:454] Detected multiple(2) devices for the segment. Picking first one to continue but this shouldn't have happened

What does this mean?
Please help.

Thanks!

2

The following message is just a warning saying that different ops in a TRT segment have different devices assigned to them. You can ignore this usually.

W tensorflow/contrib/tensorrt/convert/convert_graph.cc:454] Detected multiple(2) devices for the segment. Picking first one to continue but this shouldn't have happened

Why do you think that it couldn’t convert the graph? Have you checked the resulted graph trt_graph?