Train and convert Multioutput model into TensorRT by using TLT 2.0

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

  1. I have to train model with ‘x’ outputs . Here each output use loss ‘categorical_cross_entropy’ and activation ‘softmax’. Is this possible by using TLT?

  2. How to convert the multi output model into TensorRT?

2

I am afraid the TLT classification network meets your requirement.

3

Is that possible to convert a multioutput tensorflow trained model into TensorRT ?

4

According to TLT process, a tlt format file can be exported to etlt format file. Then convert it to trt engine.

5

We can’t train a multi output model using TLT, right ?

So I’m asking that , i have a multi output model , which is trained by using keras (not using TLT) , the weight is a .hdf5 format… is it possible to convert this multi output weight into TensorRT?

6

Yes, as of now, TLT does not support training a multi output model.
For your mentioned existing multi output model, unfortunately it is not possible to convert by using TLT.

7

But u have mentioned in the following thread that , The Tensorflow multi output model to UFF convertion is possible. Hence the UFF to TensorRT is also possible right ?

8

But TLT code does not support yet.
I will sync with internal team if this can be a feature request.

1 Like
9

Is that possible to convert Tensorflow Multioutput model to TensorRT without using TLT ?

10

Suggest your asking it in how to convert tensorflow model with mutiple output to UFF

1 Like
11

I have TRT v7.x , so i couldn’t locate the following file from this version. How to locate the file from TRT v7.x ?

Is there any alternative way to convert tf to trt ?

/usr/local/lib/python2.7/dist-packages/tensorrt/examples/tf_to_trt/tf_to_trt.py

12

@samjith888
For your questions about how to convert Tensorflow Multioutput model to TensorRT without using TLT or convert tf to trt, it is out of TLT’s scope. And also TLT code does not support it.Could you ask them in TRT forum? Thanks.

1 Like
13

You can try to convert to onnx model, then convert onnx model to trt engine.

14

I’m doing that , but i have the multi output model … I couldn’t find any example for converting the multi output model

15

Can you share the link of the 3rd multi output model? Is it a public model?

16

Sorry , Model is not public … Below is the sample model architecture

import efficientnet.tfkeras as efn
from tensorflow.keras.layers import GlobalAveragePooling2D, Dense, 
from tensorflow.keras import Model

base_model = efn.EfficientNetB4(weights='noisy-student', include_top=False)
x = base_model.output
x = GlobalAveragePooling2D()(x)
c1 = Dense(2, activation='softmax')(x)
c2 = Dense(2, activation='softmax')(x)
model = Model(inputs=base_model.input, outputs=[c1,c2])
model.compile()
model.summary()
18

Please note that currently TLT classification network does not support multioutput training.
But it is on the roadmap.