# python3: TensorRT nmsPlugin.cpp:54: error

**URL:** <https://forums.developer.nvidia.com/t/python3-tensorrt-nmsplugin-cpp-error/83749>\
**Category:** Jetson Nano\
**Created:** [October 23, 2019, 11:31am UTC](https://forums.developer.nvidia.com/t/python3-tensorrt-nmsplugin-cpp-error/83749 "2019-10-23T11:31:55Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kanakmanjari](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@kanakmanjari](https://forums.developer.nvidia.com/u/kanakmanjari)\
**Post date:** [October 23, 2019, 11:31am UTC](https://forums.developer.nvidia.com/t/python3-tensorrt-nmsplugin-cpp-error/83749/1 "2019-10-23T11:31:55Z")

</div>

I am trying to run SSdlite\_mobilenet\_v2\_cocoo on jetson nano. I was able to run ssd\_inception\_v2\_coco(2017), ssd\_mobilenet\_v1\_coco and ssd\_mobilenet\_v2\_coco successfully following [https://github.com/taka-wang/trt\_example](https://github.com/taka-wang/trt_example) repo. But when I am trying to convert .pb file of ssdlite\_mobilenet\_v2\_cocoo modell to tmp.uff and tmp.bin  
this is the error:

python3: nmsPlugin.cpp:54: virtual nvinfer1::Dims nvinfer1::plugin::DetectionOutput::getOutputDimensions(int, const nvinfer1::Dims\*, int): Assertion `nbInputDims == 3’ failed.  
Aborted (core dumped)", this error is coming…

This is file of ssdlite confuguration:

import graphsurgeon as gs

path = ‘frozen\_inference\_graph.pb’  
TRTbin = ‘tmp.bin’  
output\_name = [‘NMS’]  
dims = [3,300,300]  
layout = 7

def add\_plugin(graph):  
all\_assert\_nodes = graph.find\_nodes\_by\_op(“Assert”)  
graph.remove(all\_assert\_nodes, remove\_exclusive\_dependencies=True)

```
all_identity_nodes = graph.find_nodes_by_op("Identity")
graph.forward_inputs(all_identity_nodes)

Input = gs.create_plugin_node(
    name="Input",
    op="Placeholder",
    shape=[1, 3, 300, 300]
)

PriorBox = gs.create_plugin_node(
    name="GridAnchor",
    op="GridAnchor_TRT",
    minSize=0.2,
    maxSize=0.95,
    aspectRatios=[1.0, 2.0, 0.5, 3.0, 0.33],
    variance=[0.1,0.1,0.2,0.2],
    featureMapShapes=[19, 10, 5, 3, 2, 1],
    numLayers=6
)

NMS = gs.create_plugin_node(
    name="NMS",
    op="NMS_TRT",
    shareLocation=1,
    varianceEncodedInTarget=0,
    backgroundLabelId=0,
    confidenceThreshold=1e-8,
    nmsThreshold=0.6,
    topK=100,
    keepTopK=100,
    numClasses=91,
    inputOrder=[1, 2, 0], # its mine
    confSigmoid=1,
    isNormalized=1
)

concat_priorbox = gs.create_node(
    "concat_priorbox",
    op="ConcatV2",
    axis=2
)

concat_box_loc = gs.create_plugin_node(
    "concat_box_loc",
    op="FlattenConcat_TRT",
)

concat_box_conf = gs.create_plugin_node(
    "concat_box_conf",
    op="FlattenConcat_TRT",
)

namespace_plugin_map = {
    "MultipleGridAnchorGenerator": PriorBox,
    "Postprocessor": NMS,
    "Preprocessor": Input,
    "ToFloat": Input,
    "image_tensor": Input,
    "Concatenate": concat_priorbox,
    "concat": concat_box_loc,
    "concat_1": concat_box_conf
}

graph.collapse_namespaces(namespace_plugin_map)
graph.remove(graph.graph_outputs, remove_exclusive_dependencies=False)
graph.find_nodes_by_op("NMS_TRT")[0].input.remove("Input")

return graph

```

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [October 24, 2019, 8:25am UTC](https://forums.developer.nvidia.com/t/python3-tensorrt-nmsplugin-cpp-error/83749/2 "2019-10-24T08:25:36Z")

</div>

Hi,

The output layer between _ssdlite\_mobilenet\_v2_ and _ssd\_mobilenet\_v2_ are different.  
Please update the output name into ‘BoxPredictor\_0/ClassPredictor\_depthwise/Relu6’.

Here is the change we use for the model _ssdlite\_mobilenet\_v2_:

```auto
diff --git a/config/model_ssd_mobilenet_v2_coco_2018_03_29.py b/config/model_ssd_mobilenet_v2_coco_2018_03_29.py
index 3c9f3b8..7d8ff82 100644
--- a/config/model_ssd_mobilenet_v2_coco_2018_03_29.py
+++ b/config/model_ssd_mobilenet_v2_coco_2018_03_29.py
@@ -1,8 +1,8 @@
 import graphsurgeon as gs
 
-path = 'model/ssd_mobilenet_v2_coco_2018_03_29/frozen_inference_graph.pb'
-TRTbin = 'TRT_ssd_mobilenet_v2_coco_2018_03_29.bin'
-output_name = ['NMS']
+path = 'model/ssdlite_mobilenet_v2_coco_2018_05_09/frozen_inference_graph.pb'
+TRTbin = 'TRT_ssdlite_mobilenet_v2_coco_2018_05_09.bin'
+output_name = ['BoxPredictor_0/ClassPredictor_depthwise/Relu6']
 dims = [3,300,300]
 layout = 7

```

Thanks.

---

<div class="post-metadata">

**Author:** ![kanakmanjari](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@kanakmanjari](https://forums.developer.nvidia.com/u/kanakmanjari)\
**Post date:** [October 25, 2019, 5:47am UTC](https://forums.developer.nvidia.com/t/python3-tensorrt-nmsplugin-cpp-error/83749/3 "2019-10-25T05:47:12Z")

</div>

I tried what you have suggested but while converting below is the trace back comes at end. whilefFor other three models mentioned in my previous post, it run successfully.

Traceback (most recent call last):  
File “convert.py”, line 88, in   
cuda.memcpy\_dtoh\_async(host\_outputs[1], cuda\_outputs[1], stream)  
IndexError: list index out of range

Below is the convert.py file :

import os  
import sys  
import cv2  
import time  
import ctypes  
import numpy as np  
import pycuda.autoinit  
import pycuda.driver as cuda

import uff  
import tensorrt as trt  
import graphsurgeon as gs  
#from config import model\_ssd\_inception\_v2\_coco\_2017\_11\_17 as model  
#from config import model\_ssd\_mobilenet\_v1\_coco\_2018\_01\_28 as model  
#from config import model\_ssd\_mobilenet\_v2\_coco\_2018\_03\_29 as model  
from config import model\_ssdlite\_mobilenet\_v2\_coco\_2018\_05\_09 as model  
import utils.coco as coco\_utils # COCO dataset descriptors  
COCO\_LABELS = coco\_utils.COCO\_CLASSES\_LIST # change this

ctypes.CDLL(“lib/libflattenconcat.so”)

# initialize

TRT\_LOGGER = trt.Logger(trt.Logger.INFO)  
trt.init\_libnvinfer\_plugins(TRT\_LOGGER, ‘’)  
runtime = trt.Runtime(TRT\_LOGGER)

# compile model into TensorRT

if not os.path.isfile(model.TRTbin):  
dynamic\_graph = model.add\_plugin(gs.DynamicGraph(model.path))  
uff\_model = uff.from\_tensorflow(dynamic\_graph.as\_graph\_def(), model.output\_name, output\_filename=‘tmp.uff’)

```
with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.UffParser() as parser:
    builder.max_workspace_size = 1 << 28
    builder.max_batch_size = 1
    builder.fp16_mode = True

    parser.register_input('Input', model.dims)
    parser.register_output('MarkOutput_0')
    parser.parse('tmp.uff', network)
    engine = builder.build_cuda_engine(network)

    buf = engine.serialize()
    with open(model.TRTbin, 'wb') as f:
        f.write(buf)

```

# create engine

with open(model.TRTbin, ‘rb’) as f:  
buf = f.read()  
engine = runtime.deserialize\_cuda\_engine(buf)

# create buffer

host\_inputs =   
cuda\_inputs =   
host\_outputs =   
cuda\_outputs =   
bindings =   
stream = cuda.Stream()

for binding in engine:  
size = trt.volume(engine.get\_binding\_shape(binding)) \* engine.max\_batch\_size  
host\_mem = cuda.pagelocked\_empty(size, np.float32)  
cuda\_mem = cuda.mem\_alloc(host\_mem.nbytes)

```
bindings.append(int(cuda_mem))
if engine.binding_is_input(binding):
    host_inputs.append(host_mem)
    cuda_inputs.append(cuda_mem)
else:
    host_outputs.append(host_mem)
    cuda_outputs.append(cuda_mem)

```

context = engine.create\_execution\_context()

# inference

#TODO enable video pipeline  
#TODO using pyCUDA for preprocess  
ori = cv2.imread(sys.argv[1])  
image = cv2.cvtColor(ori, cv2.COLOR\_BGR2RGB)  
image = cv2.resize(image, (model.dims[2],model.dims[1]))  
image = (2.0/255.0) \* image - 1.0  
image = image.transpose((2, 0, 1))  
np.copyto(host\_inputs[0], image.ravel())

start\_time = time.time()  
cuda.memcpy\_htod\_async(cuda\_inputs[0], host\_inputs[0], stream)  
context.execute\_async(bindings=bindings, stream\_handle=stream.handle)  
cuda.memcpy\_dtoh\_async(host\_outputs[1], cuda\_outputs[1], stream)  
cuda.memcpy\_dtoh\_async(host\_outputs[0], cuda\_outputs[0], stream)  
stream.synchronize()  
print("execute times "+str(time.time()-start\_time))

output = host\_outputs[0]  
height, width, channels = ori.shape  
for i in range(int(len(output)/model.layout)):  
prefix = i\*model.layout  
index = int(output[prefix+0])  
label = int(output[prefix+1])  
conf = output[prefix+2]  
xmin = int(output[prefix+3]\*width)  
ymin = int(output[prefix+4]\*height)  
xmax = int(output[prefix+5]\*width)  
ymax = int(output[prefix+6]\*height)

```
if conf > 0.7:
    print("Detected {} with confidence {}".format(COCO_LABELS[label], "{0:.0%}".format(conf)))
    cv2.rectangle(ori, (xmin,ymin), (xmax, ymax), (0,0,255),3)
    cv2.putText(ori, COCO_LABELS[label],(xmin+10,ymin+10), cv2.FONT_HERSHEY_SIMPLEX, 1, (255,255,255), 2, cv2.LINE_AA)

```

cv2.imwrite(“result.jpg”, ori)  
cv2.imshow(“result”, ori)  
cv2.waitKey(0)

---

<div class="post-metadata">

**Author:** ![LoveNvidia](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@LoveNvidia](https://forums.developer.nvidia.com/u/LoveNvidia)\
**Post date:** [January 4, 2020, 8:30am UTC](https://forums.developer.nvidia.com/t/python3-tensorrt-nmsplugin-cpp-error/83749/4 "2020-01-04T08:30:11Z")

</div>

Hi @kanakmanjari,  
I encountered the same error, Can you solve the problem?

---

<div class="post-metadata">

**Author:** ![zryang\_ok](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@zryang\_ok](https://forums.developer.nvidia.com/u/zryang_ok)\
**Post date:** [July 15, 2020, 2:08am UTC](https://forums.developer.nvidia.com/t/python3-tensorrt-nmsplugin-cpp-error/83749/5 "2020-07-15T02:08:51Z")

</div>

I followed your change for the model ssdlite\_mobilenet\_v2. But I got the wrong result below, can you check it? thanks!

```auto
=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: 1
      }
      dim {
        size: 3
      }
      dim {
        size: 300
      }
      dim {
        size: 300
      }
    }
  }
}
]
=========================================

Using output node BoxPredictor_0/ClassPredictor_depthwise/Relu6
Converting to UFF graph
DEBUG [/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py:96] Marking ['BoxPredictor_0/ClassPredictor_depthwise/Relu6'] as outputs
No. nodes: 231
UFF Output written to tmp.uff
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[TensorRT] INFO: Detected 1 inputs and 1 output network tensors.
infer times 0.00864863395690918
Detected bicycle with confidence 84%
Detected unlabeled with confidence 79%
Detected unlabeled with confidence 138%
Detected unlabeled with confidence 81%
Detected unlabeled with confidence 147%
Detected unlabeled with confidence 171%
Detected unlabeled with confidence 87%
Detected unlabeled with confidence 234%
Detected unlabeled with confidence 97%
Detected unlabeled with confidence 167%
Detected unlabeled with confidence 130%
Detected unlabeled with confidence 113%
Detected unlabeled with confidence 92%
Detected unlabeled with confidence 84%
Detected unlabeled with confidence 314%
Detected unlabeled with confidence 121%
Detected unlabeled with confidence 123%
Detected unlabeled with confidence 117%
Detected unlabeled with confidence 117%
Detected car with confidence 130%
Detected unlabeled with confidence 83%
Detected unlabeled with confidence 107%
Detected unlabeled with confidence 306%
Detected unlabeled with confidence 180%
Detected unlabeled with confidence 153%
Detected unlabeled with confidence 145%
Detected unlabeled with confidence 162%
Detected unlabeled with confidence 178%
Detected unlabeled with confidence 326%
Detected unlabeled with confidence 71%
Detected unlabeled with confidence 319%
Detected person with confidence 252%
Detected unlabeled with confidence 111%
Detected unlabeled with confidence 358%
Detected unlabeled with confidence 74%
Detected unlabeled with confidence 88%
Detected unlabeled with confidence 137%
Detected unlabeled with confidence 149%
Detected unlabeled with confidence 114%
Detected unlabeled with confidence 244%
Detected unlabeled with confidence 145%
Detected unlabeled with confidence 104%
Detected unlabeled with confidence 95%
Detected unlabeled with confidence 98%
Detected unlabeled with confidence 167%
Detected unlabeled with confidence 82%
Detected unlabeled with confidence 216%
Detected unlabeled with confidence 98%
Detected unlabeled with confidence 165%
Detected unlabeled with confidence 100%
Detected unlabeled with confidence 106%
Detected unlabeled with confidence 157%
Detected unlabeled with confidence 163%
Detected unlabeled with confidence 194%
Detected unlabeled with confidence 119%
Detected unlabeled with confidence 158%
Detected unlabeled with confidence 155%
Detected unlabeled with confidence 74%
Detected unlabeled with confidence 136%
Detected unlabeled with confidence 115%
Detected unlabeled with confidence 76%
Detected unlabeled with confidence 113%
Detected unlabeled with confidence 74%
Detected unlabeled with confidence 96%
Detected person with confidence 90%
Detected unlabeled with confidence 268%
Detected unlabeled with confidence 173%
Detected unlabeled with confidence 136%
Detected unlabeled with confidence 74%
Detected unlabeled with confidence 72%
Detected unlabeled with confidence 78%
Detected unlabeled with confidence 79%
Detected unlabeled with confidence 82%
Detected unlabeled with confidence 263%
Detected unlabeled with confidence 161%
Detected unlabeled with confidence 76%
Detected unlabeled with confidence 110%
Detected unlabeled with confidence 90%
Detected unlabeled with confidence 134%
Detected unlabeled with confidence 185%
Detected unlabeled with confidence 235%
Detected unlabeled with confidence 135%
Detected unlabeled with confidence 245%
Detected unlabeled with confidence 112%
Detected unlabeled with confidence 280%
Detected unlabeled with confidence 164%
Detected unlabeled with confidence 186%
Detected unlabeled with confidence 71%
Detected unlabeled with confidence 96%
Detected unlabeled with confidence 109%
Detected unlabeled with confidence 99%
Detected unlabeled with confidence 205%
Detected unlabeled with confidence 72%
Detected unlabeled with confidence 236%
Detected unlabeled with confidence 146%
Detected unlabeled with confidence 163%
Detected unlabeled with confidence 143%
Detected unlabeled with confidence 104%
Detected unlabeled with confidence 110%
Detected unlabeled with confidence 144%
Detected unlabeled with confidence 79%
Detected unlabeled with confidence 174%
Detected unlabeled with confidence 140%
Detected unlabeled with confidence 82%
Detected unlabeled with confidence 217%
Detected unlabeled with confidence 82%
Detected unlabeled with confidence 128%
Detected unlabeled with confidence 200%
Detected unlabeled with confidence 318%
Detected unlabeled with confidence 94%
Detected unlabeled with confidence 84%
Detected unlabeled with confidence 159%
Detected unlabeled with confidence 149%
Detected unlabeled with confidence 108%
Detected person with confidence 76%
Detected unlabeled with confidence 81%
Detected unlabeled with confidence 86%
Detected unlabeled with confidence 310%
Detected unlabeled with confidence 103%
Detected unlabeled with confidence 133%
Detected unlabeled with confidence 120%
Detected unlabeled with confidence 74%
Detected unlabeled with confidence 259%
Detected unlabeled with confidence 93%
Detected unlabeled with confidence 162%
Detected person with confidence 230%
Detected unlabeled with confidence 97%
Detected unlabeled with confidence 237%
Detected unlabeled with confidence 99%
Detected unlabeled with confidence 119%
Detected unlabeled with confidence 90%
Detected unlabeled with confidence 180%
Detected unlabeled with confidence 202%
Detected unlabeled with confidence 101%
Detected unlabeled with confidence 76%
Detected unlabeled with confidence 100%
Detected unlabeled with confidence 220%
Detected unlabeled with confidence 82%
Detected unlabeled with confidence 214%
Detected unlabeled with confidence 199%
Detected unlabeled with confidence 316%
Detected unlabeled with confidence 82%
Detected unlabeled with confidence 88%
Detected unlabeled with confidence 264%
Detected unlabeled with confidence 242%
Detected unlabeled with confidence 161%
Detected unlabeled with confidence 135%
Detected unlabeled with confidence 81%
Detected unlabeled with confidence 97%
Detected unlabeled with confidence 208%
Detected unlabeled with confidence 84%
Detected unlabeled with confidence 106%
Detected unlabeled with confidence 168%
Detected unlabeled with confidence 122%
Detected unlabeled with confidence 103%
Detected unlabeled with confidence 71%
Detected unlabeled with confidence 102%
Detected unlabeled with confidence 99%
Detected unlabeled with confidence 88%
Detected unlabeled with confidence 83%
Detected unlabeled with confidence 126%
Detected unlabeled with confidence 384%
Detected unlabeled with confidence 234%
Detected unlabeled with confidence 76%
Detected unlabeled with confidence 116%
Detected unlabeled with confidence 275%
Detected unlabeled with confidence 116%
Detected unlabeled with confidence 308%
Detected unlabeled with confidence 265%
Detected unlabeled with confidence 121%
Detected unlabeled with confidence 145%
Detected unlabeled with confidence 153%
Detected unlabeled with confidence 193%
Detected unlabeled with confidence 131%
Detected unlabeled with confidence 77%
Detected unlabeled with confidence 153%
Detected unlabeled with confidence 99%
Detected unlabeled with confidence 76%
Detected unlabeled with confidence 73%
Detected unlabeled with confidence 166%
Detected unlabeled with confidence 174%
Detected unlabeled with confidence 97%
Detected bicycle with confiden

```

---

<div class="post-metadata">

**Author:** ![nadeemm](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nadeemm/32/367020_2.png) [@nadeemm](https://forums.developer.nvidia.com/u/nadeemm)\
**Post date:** [October 14, 2021, 9:59pm UTC](https://forums.developer.nvidia.com/t/python3-tensorrt-nmsplugin-cpp-error/83749/6 "2021-10-14T21:59:59Z")

</div>

Closed on October 14, 2021, 9:59pm UTC


