Description
Hi, I am trying to do inference using TensorRT 7.0 on a network (basically VGG16 with some attention layers to do semantic segmentation) trained in tensorflow 1.14 and got error message “UffParser: Output error: Output out not found”. I could not figure out what is wrong with my implementation.
For now, I am not using CMAKE and compile code with “sudo g++ -g -o fs infer.cpp logger.cpp -I/usr/local/cuda/include -L/usr/local/cuda/lib64 -lcudart -lcuda -L/usr/lib/x86_64-linux-gnu -lnvinfer -lnvparsers”.
When I try to run the code with “./fs --fp16” I got an output as below:
&&&& RUNNING TensorRT.FreeSpaceUff # ./fs --fp16
[05/28/2020-13:15:38] [I] Building and running a GPU inference engine for Uff Free Space
[05/28/2020-13:15:38] [E] [TRT] UffParser: Output error: Output out not found
[05/28/2020-13:15:39] [I] [TRT] Detected 0 inputs and 1 output network tensors.
Segmentation fault (core dumped)
Does anyone have clue about this error? Is there any possibility that this error caused by constructing uff model or tf frozen graph wrongly?
Environment
TensorRT Version: 7.0.0
GPU Type: Quadro RTX 5000
Nvidia Driver Version: 440.33.01
CUDA Version: 10.2
CUDNN Version: 7.6.5
Operating System + Version: Ubuntu 18.04
Based on the error it seems that the output name fed into the application does not exist in the model file you loaded.
Could you please check if provided output name exist in the model?
Thanks
Thanks for your reply. After correcting output name this error resolved.
Now I think, I have the same problem with input since 0 inputs detected.
But I have no idea about the input tensor name right now. I have the tf checkpoints for pretrained model. I have tried to load it tensorboard to see input tensor name but I couldn’t find it there. Can you give me any idea how to find input tensor name?
When you click a node in the TensorBoard graph some information about that node will be displayed at the corner top right. Along with the node name other details will also be displayed.
By selecting the input tensor node you can get all the details related to that node.
Thanks
Yes I have tried it but still having the same problem. I also have .pbtxt file for the model I have checked input from there too. I can put the some part of the .pbtxt file here to get your ideas.
node {
name: "Placeholder"
op: "Placeholder"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
dim {
size: 1
}
dim {
size: 480
}
dim {
size: 760
}
dim {
size: 3
}
}
}
}
}
node {
name: "preprocess/img_mean"
op: "Const"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "value"
value {
tensor {
dtype: DT_FLOAT
tensor_shape {
dim {
size: 1
}
dim {
size: 1
}
dim {
size: 1
}
dim {
size: 3
}
}
tensor_content: ")\\\367B\331\216\351B\305\340\317B"
}
}
}
}
node {
name: "preprocess/sub"
op: "Sub"
input: "Placeholder"
input: "preprocess/img_mean"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
}
node {
name: "conv1_1/truncated_normal/shape"
op: "Const"
attr {
key: "dtype"
value {
type: DT_INT32
}
}
attr {
key: "value"
value {
tensor {
dtype: DT_INT32
tensor_shape {
dim {
size: 4
}
}
tensor_content: "\003\000\000\000\003\000\000\000\003\000\000\000@\000\000\000"
}
}
}
}
node {
name: "conv1_1/truncated_normal/mean"
op: "Const"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "value"
value {
tensor {
dtype: DT_FLOAT
tensor_shape {
}
float_val: 0.0
}
}
}
}
node {
name: "conv1_1/truncated_normal/stddev"
op: "Const"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "value"
value {
tensor {
dtype: DT_FLOAT
tensor_shape {
}
float_val: 0.10000000149011612
}
}
}
}
node {
name: "conv1_1/truncated_normal/TruncatedNormal"
op: "TruncatedNormal"
input: "conv1_1/truncated_normal/shape"
attr {
key: "T"
value {
type: DT_INT32
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "seed"
value {
i: 0
}
}
attr {
key: "seed2"
value {
i: 0
}
}
}
node {
name: "conv1_1/truncated_normal/mul"
op: "Mul"
input: "conv1_1/truncated_normal/TruncatedNormal"
input: "conv1_1/truncated_normal/stddev"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
}
node {
name: "conv1_1/truncated_normal"
op: "Add"
input: "conv1_1/truncated_normal/mul"
input: "conv1_1/truncated_normal/mean"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
}
node {
name: "conv1_1/weights"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
dim {
size: 3
}
dim {
size: 3
}
dim {
size: 3
}
dim {
size: 64
}
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}
}
node {
name: "conv1_1/weights/Assign"
op: "Assign"
input: "conv1_1/weights"
input: "conv1_1/truncated_normal"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
attr {
key: "_class"
value {
list {
s: "loc:@conv1_1/weights"
}
}
}
attr {
key: "use_locking"
value {
b: true
}
}
attr {
key: "validate_shape"
value {
b: true
}
}
}
node {
name: "conv1_1/weights/read"
op: "Identity"
input: "conv1_1/weights"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
attr {
key: "_class"
value {
list {
s: "loc:@conv1_1/weights"
}
}
}
}
node {
name: "conv1_1/Conv2D"
op: "Conv2D"
input: "preprocess/sub"
input: "conv1_1/weights/read"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
attr {
key: "data_format"
value {
s: "NHWC"
}
}
attr {
key: "dilations"
value {
list {
i: 1
i: 1
i: 1
i: 1
}
}
}
attr {
key: "explicit_paddings"
value {
list {
}
}
}
attr {
key: "padding"
value {
s: "SAME"
}
}
attr {
key: "strides"
value {
list {
i: 1
i: 1
i: 1
i: 1
}
}
}
attr {
key: "use_cudnn_on_gpu"
value {
b: true
}
}
}
node {
name: "conv1_1/Const"
op: "Const"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "value"
value {
tensor {
dtype: DT_FLOAT
tensor_shape {
dim {
size: 64
}
}
float_val: 0.0
}
}
}
}
node {
name: "conv1_1/biases"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
dim {
size: 64
}
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}
}
node {
name: "conv1_1/biases/Assign"
op: "Assign"
input: "conv1_1/biases"
input: "conv1_1/Const"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
attr {
key: "_class"
value {
list {
s: "loc:@conv1_1/biases"
}
}
}
attr {
key: "use_locking"
value {
b: true
}
}
attr {
key: "validate_shape"
value {
b: true
}
}
}
node {
name: "conv1_1/biases/read"
op: "Identity"
input: "conv1_1/biases"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
attr {
key: "_class"
value {
list {
s: "loc:@conv1_1/biases"
}
}
}
}
node {
name: "conv1_1/BiasAdd"
op: "BiasAdd"
input: "conv1_1/Conv2D"
input: "conv1_1/biases/read"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
attr {
key: "data_format"
value {
s: "NHWC"
}
}
}
node {
name: "conv1_1"
op: "Relu"
input: "conv1_1/BiasAdd"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
}
node {
name: "conv1_2/truncated_normal/shape"
op: "Const"
attr {
key: "dtype"
value {
type: DT_INT32
}
}
attr {
key: "value"
value {
tensor {
dtype: DT_INT32
tensor_shape {
dim {
size: 4
}
}
tensor_content: "\003\000\000\000\003\000\000\000@\000\000\000@\000\000\000"
}
}
}
}
node {
name: "conv1_2/truncated_normal/mean"
op: "Const"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "value"
value {
tensor {
dtype: DT_FLOAT
tensor_shape {
}
float_val: 0.0
}
}
}
}
Can you share the script and model file to reproduce the issue so we can help better?
Meanwhile, can you try using trtexec command line tool?
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
Thanks
I will send the scripts and model to you and I will try trtexec.
Thanks
Can you try using following command:
trtexec --uff=/test/network_model.uff --uffInput=Placeholder:0,3,480,760 --output=Assign_25/value --verbose --uffNHWC
It seems to be working on my system, I think input value you can use in this case is Placeholder:0 or Placeholder
[06/01/2020-10:36:02] [I] total compute time: 0.503205 s
&&&& PASSED TensorRT.trtexec # ./trtexec --uff=/test/network_model.uff --uffInput=Placeholder:0,3,480,760 --output=Assign_25/value --verbose --uffNHWC
Thanks