converting UFF model to tensorrt

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I have read the guide on converting TensorFlow model to tensorRT: https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#import_tf_python. In the last step, step 5, the code example is given as:

with builder = trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.UffParser() as parser:
    	parser.register_input("Placeholder", (1, 28, 28))
    	parser.register_output("fc2/Relu")
parser.parse(model_file, network)

  1. I have looked up UFF parser documentation but it is still not clear to me what the input to “register_output” must be. can it be any name – like how we create variable names – or must it be from a specific list of valid operation names?

  2. for “register_input(name, shape, order)”, what must the name of the input be? can it be any name that I chose fit or must it be a specific name of a node? And is there a way to make the shape of the input variable? For instance, if I am trying to convert speech to text or text to speech models to TensorRt, the inputs to STT and TTS is going to be variable.

2

The input to “register_output” is the output of your model, as the name already suggests. Often it is called “classifier”, “fc/classifier” or whatever. Check with tensorboard the structure of your model, so you find out what the real name of your output is.

At least that’s what i found out so far.
Wish you good luck.

3

Hi,
If the input size of the image is fixed, for example
int C=3;
int H=224;
int W=224;
parser->registerInput(“input”, DimsCHW(C, H, W), UffInputOrder::kNCHW);
it can run normally.

If the input size of the image is not fixed, for example C=3, H=?, W=?
How to express parser->registerInput()?
Does tensoeRT support formats with variable input sizes?

Thank you very much!