The difference between TF and TRT weights?

Hi, I create a network with TRT API, and load tf weights into the TRT network, but the results are different.
I think the weight format difference may lead to the problem. So i have some questions to ask:

(1) for the conv weights, TF uses RSCK ([filter_height, filter_width, input_depth, output_depth]) and TensorRT uses KCRS, so do i need to transpose the weights before feeding to the conv layers? but in the SAMPLEMNISTAPI, there is no tranpose to tf weights for TRT.
for my example:
w1 = weightMap[“tf_conv1”] #RSCK
transpose the tf weights???
conv1 = network.add_convolution(data, 32, (11,41), trt.infer.Weights(w1), b1)

(2) for the IScaleLayer, I set tensorrt.ScaleMode.CHANNEL. The TRT doc says that the channel dimension is assumed to be the third to last dimension, but TRT format is CHW, so do i need to transpose the input of ScaleLayer from CHW to HWC?

(3) for RNN layer, TRT docs says that tf weights format is data_lenhidden_size, while TRT expects hidden_sizedata_len, so should i need to transpose each gate weights of RNN cell?

(4) recently, tf can set data format like channel_first(NCHW) and channel_last(NHWC), so if I train a tf model with data_format=channel_first, are the tf weights format the same as TRT weights format? If so, does it mean I don’t need to convert TF weights for TRT?


For TensorFlow, the uff parser automatically performs the required format handling.
Note: no matter the model is NCHW or NHWC, remember to register your input blob with NCHW format.

You can find detail information in our document:

Hi, what i mean is that I create a network definition from scratch using the python TRT API, and load TF weights to it like:

w1 = weightMap[CONV1_CONVW]
b1 = trt.infer.Weights.empty(trt.infer.DataType.FLOAT)
conv1 = network.add_convolution(data, 32, (11,41), trt.infer.Weights(w1), b1)

and the TF weight format is different from TRT format.

So I wonder whether i need to convert TF weights (w1) to TRT format , then feed to the network.add_convolution API?

Hi @jianxiangm,

did you test transposing the weight format yet? I am doing something similar here: coding the network definition using C++ API and loading the convolutional and bias weights from a trained .h5 file from keras. The C++ code would give me totally different results, even with only one conv layer.

I am also concerned whether the weight format could be an issue and am about to test it out. But i am wondering whether you already tested this?

Thank you very much!

The tf weights format is RSCK, but trt weights format is KCRS, so you need to transpose the weights for conv layers.

Hi @jianxiangm,

thank you very much for the reply! I transposed the weight format and now my c++ network is able to do inference.

Thanks again for your post in the first place! I didn’t find anything related to weight format in the latest documentation. But from your post, I was able to find documentation for TRT-2.1 or something like that and get a better hold of the issue.