the converted result of softmax in tensorflow is not correct

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

I convert a tensorflow model to UFF. The output of uff is the same as the original one before softmax operation. Howerver, it become different after the softmax operation. It seems that tensorrt does not convert the tensorflow softmax operation rightly. My graph is as below:
…
pre_class = tf.reshape(conv_class, [-1, classes_])(the shape of conv_class is [batchsize, height, width, 5*classes_])
pre_class = tf.nn.softmax(pre_class)

2

Hi,

Softmax operation requires proper input order to have reasonable behavior.
Currently, we apply softmax against channel (NCHW).

But your format is NHWC, which will yield the incorrect results.

Thanks.

3

Hi AastaLLL:
I change my network to NCHW,

the shape of conv_class is [batchsize, 5classes_, height, width], but I want to do softmax on classes, not on 5classes.I add the operation as below:

pre_class = tf.reshape(conv_class, [batchsize*5,classes_, height,width])
pre_class = tf.transpose(pre_class, perm=[0,2,3,1])
pre_class = tf.reshape(conv_class, [-1, classes_])
pre_class = tf.nn.softmax(pre_class)

howerver, it do softmax on 5*classes.how could I do softmax on classes?

4

Hi,

If your data arrangement is c0, c1, c2,…, c0, c1, c2, …, try this:

output = tf.reshape(inputs, [batchsize*5, classes, height, width])
output = tf.nn.softmax(output, dim=1)
output = tf.reshape(output, [batchsize, 5*classes, height, width])

Thanks

5

Hi AastaLLL:

I did as what you told, but the function output = tf.nn.softmax(output, dim=1) makes the model fail to convert to UFF

uff_model = uff.from_tensorflow(frozen_graph, OUTPUT_NAMES, output_filename=UFF_OUTPUT_FILENAME, text=True)
File “/usr/local/lib/python2.7/dist-packages/uff/converters/tensorflow/conversion_helpers.py”, line 75, in from_tensorflow
name=“main”)
File “/usr/local/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py”, line 64, in convert_tf2uff_graph
uff_graph, input_replacements)
File “/usr/local/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py”, line 51, in convert_tf2uff_node
op, name, tf_node, inputs, uff_graph, tf_nodes=tf_nodes)
File “/usr/local/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py”, line 32, in convert_layer
return cls.registry_[op](name, tf_node, inputs, uff_graph, **kwargs)
File “/usr/local/lib/python2.7/dist-packages/uff/converters/tensorflow/converter_functions.py”, line 150, in convert_transpose
tf_permutation_node).tolist()
File “/usr/local/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py”, line 86, in convert_tf2numpy_const_node
np_dtype = cls.convert_tf2numpy_dtype(tf_node.attr[‘dtype’].type)
File “/usr/local/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py”, line 74, in convert_tf2numpy_dtype
return np.dtype(dt[dtype])
TypeError: data type “invalid” not understood

Thanks

6

Hello,

I am also trying to convert a frozen tensorflow graph with a softmax layer at the end to UFF.

However, I didn’t manage yet because several operations aren’t supported by Tensor RT. They are :

  • Max
  • RealDiv or truediv
  • _Slice if I used directly tf.nn.softmax(tensor, dim=-1)

Do you have any suggestion for implementing something compatible with Tensor RT for Softmax ?

Thank you very much,

Denys

7

Hi,

Thank for the feedback.
We are checking this issue internally. Will update information here.

Thanks.

8

Hi,

This is a known issue and is fixed in our next TensorRT release.
Thanks and sorry for the inconvenience.

9

Hello,

Ok thank you. Could we know when you approximately plan to release this new version ?

Thanks

10

Hi,

Sorry for that we can’t disclosure our schedule.
Please pay attention to official announcement and update.

Thanks.

11

Hi there,

Has this been fixed in the latest TensorRT 5.0 release? I still came across the same issue in my test.

Thanks.

12

Hi,

It’s solved in TensorRT 5.0.

Also share our testing sample here:

import tensorflow as tf
import numpy as np
import uff

batchsize = 1
height = 1
width = 1
classes = 3

inputs = tf.placeholder(tf.float32, shape=(batchsize, height, width, 5*classes))
output = tf.reshape(inputs, [-1,classes])
output = tf.nn.softmax(output, name='output')

with tf.Session() as sess:
    graphdef = tf.get_default_graph().as_graph_def()
    frozen_graph = tf.graph_util.convert_variables_to_constants(sess, graphdef, ['output'])
    tf_model = tf.graph_util.remove_training_nodes(frozen_graph)

uff_model = uff.from_tensorflow(tf_model, ["output"])

Thanks.

13

Are you serious? What the heck is the reshape before softmax?

The following case should anyway be covered and tested to call it a “fix”. At least document it somewhere otherwise the users won’t know.

inputs = tf.placeholder(tf.float32, shape=(batchsize, height, width, 5*classes))
output = tf.nn.softmax(intputs, axis=-1, name='output')
with tf.Session() as sess:
    graphdef = tf.get_default_graph().as_graph_def()
    frozen_graph = tf.graph_util.convert_variables_to_constants(sess, graphdef, ['output'])
    tf_model = tf.graph_util.remove_training_nodes(frozen_graph)
uff_model = uff.from_tensorflow(tf_model, ["output"])

Traceback (most recent call last):
File “test_uff.py”, line 26, in
uff_model = uff.from_tensorflow(tf_model, [“output”])
File “/usr/local/lib/python2.7/site-packages/uff/converters/tensorflow/conversion_helpers.py”, line 132, in from_tensorflow
name=“main”)
File “/usr/local/lib/python2.7/site-packages/uff/converters/tensorflow/converter.py”, line 77, in convert_tf2uff_graph
uff_graph, input_replacements)
File “/usr/local/lib/python2.7/site-packages/uff/converters/tensorflow/converter.py”, line 54, in convert_tf2uff_node
raise UffException(str(name) + " was not found in the graph. Please use the -l option to list nodes in the graph.")
uff.model.exceptions.UffException: output was not found in the graph. Please use the -l option to list nodes in the graph.

14

Hi,

Could you share more information with your issue?
Do you meet any error with TensorRT-5.0?

We can convert the model into uff without any error.

inputs = tf.placeholder(tf.float32, shape=(batchsize, height, width, 5*classes))
output = tf.nn.softmax(inputs, name='output')

The reshape layer is not necessary here.
It was added for covering the user’s use case.

Thanks.