# Unable to create plugin for BatchedNMSDynamic\_TRT for onnx tao model using graphsurgeon **URL:** \ **Category:** TAO Toolkit\ **Created:** [July 18, 2024, 11:22am UTC](https://forums.developer.nvidia.com/t/unable-to-create-plugin-for-batchednmsdynamic-trt-for-onnx-tao-model-using-graphsurgeon/300267 "2024-07-18T11:22:26Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![sarvatmak](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@sarvatmak](https://forums.developer.nvidia.com/u/sarvatmak)\ **Post date:** [July 18, 2024, 11:22am UTC](https://forums.developer.nvidia.com/t/unable-to-create-plugin-for-batchednmsdynamic-trt-for-onnx-tao-model-using-graphsurgeon/300267/1 "2024-07-18T11:22:27Z")
I have created TAO model yolov4. It is onnx format but i am not able to run it using onnxruntime-gpu so i found that it contains custome layers, i reffered following links:- [BatchedNMS\_TRT: could not find any supported formats consistent with input/output data types · Issue #954 · NVIDIA/TensorRT · GitHub](https://github.com/NVIDIA/TensorRT/issues/954) [BatchedNMS\_TRT: could not find any supported formats consistent with input/output data types · Issue #954 · NVIDIA/TensorRT · GitHub](https://github.com/NVIDIA/TensorRT/issues/954) so i wrote the following code snippet * * * > import onnx\_graphsurgeon as gs > import numpy as np > import onnx > graph = gs.import\_onnx(onnx.load(“yolov4.onnx”)) > @gs.Graph.register() > def trt\_batched\_nms(self, boxes\_input, scores\_input, nms\_output, > shareLocation, numClasses): # and other attrs > boxes\_input.outputs.clear() > scores\_input.outputs.clear() > nms\_output.inputs.clear() > attrs = { > “shareLocation”: shareLocation, > “numClasses” : numClasses, > “backgroundLabelId”: -1, > “topK”:400, > “keepTopK”:200, > “scoreBits”:0, > “scoreThreshold”:0.001, > “iouThreshold”: 0.5, > “isNormalized”:True, > “clipBoxes”:True > } > return self.layer(op=“BatchedNMSDynamic\_TRT”, attrs=attrs, > inputs=[boxes\_input, scores\_input], > outputs=[nms\_output]) > tmap = graph.tensors() > graph.trt\_batched\_nms(tmap[“box”], tmap[“cls”], > tmap[“BatchedNMS”], shareLocation = True, > numClasses = 2) > graph.trt\_batched\_nms(tmap[“box”], tmap[“cls”], > tmap[“BatchedNMS\_1”], shareLocation = True, > numClasses = 2) > graph.trt\_batched\_nms(tmap[“box”], tmap[“cls”], > tmap[“BatchedNMS\_2”], shareLocation = True, > numClasses = 2) > graph.trt\_batched\_nms(tmap[“box”], tmap[“cls”], > tmap[“BatchedNMS\_3”], shareLocation = True, > numClasses = 2) > graph.cleanup().toposort() > graph.fold\_constants().cleanup() > onnx.checker.check\_model(gs.export\_onnx(graph)) * * * but i get following error [W] Inference failed. You may want to try enabling partitioning to see better results. Note: Error was: [ONNXRuntimeError] : 10 : INVALID\_GRAPH : This is an invalid model. In Node, (“onnx\_graphsurgeon\_node\_0”, BatchedNMSDynamic\_TRT, “”, -1) : () → (“BatchedNMS”: tensor(int32),) , Error No Op registered for BatchedNMSDynamic\_TRT with domain\_version of 12 * * * ValidationError Traceback (most recent call last) Cell In[14], line 49 47 graph.fold\_constants().cleanup() 48 # print(tmap) —\> 49 onnx.checker.check\_model(gs.export\_onnx(graph)) File ~\AppData\Local\anaconda3\envs\gpu\_env\_demo\lib\site-packages\onnx\checker.py:179, in check\_model(model, full\_check, skip\_opset\_compatibility\_check, check\_custom\_domain) 175 if sys.getsizeof(protobuf\_string) \> MAXIMUM\_PROTOBUF: 176 raise ValueError( 177 “This protobuf of onnx model is too large (\>2GB). Call check\_model with model path instead.” 178 ) → 179 C.check\_model( 180 protobuf\_string, 181 full\_check, 182 skip\_opset\_compatibility\_check, 183 check\_custom\_domain, 184 ) ValidationError: Field ‘shape’ of ‘type’ is required but missing. * * * the shapes are dynamic so none * * * out\_tensors = graph.outputs print(out\_tensors) [Variable (BatchedNMS): (shape=None, dtype=int32), Variable (BatchedNMS\_1): (shape=None, dtype=float32), Variable (BatchedNMS\_2): (shape=None, dtype=float32), Variable (BatchedNMS\_3): (shape=None, dtype=float32)] * * * I feel like i am missing something can anyone please help me out. i have alredy created engine for triton on jetson using trtexec. Now i need it to run with gpu onnx runtime. ---
**Author:** ![Morganh](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/morganh/32/9748_2.png) [@Morganh](https://forums.developer.nvidia.com/u/Morganh)\ **Post date:** [July 18, 2024, 4:52pm UTC](https://forums.developer.nvidia.com/t/unable-to-create-plugin-for-batchednmsdynamic-trt-for-onnx-tao-model-using-graphsurgeon/300267/2 "2024-07-18T16:52:44Z")
> [@sarvatmak](#): > > Now i need it to run with gpu onnx runtime. Similar to topic [Errors while reading ONNX file produced by TAO 5 - #10 by Morganh](https://forums.developer.nvidia.com/t/errors-while-reading-onnx-file-produced-by-tao-5/264792/10). ---
**Author:** ![yingliu](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/yingliu/32/136703_2.png) [@yingliu](https://forums.developer.nvidia.com/u/yingliu)\ **Post date:** [August 16, 2024, 8:16am UTC](https://forums.developer.nvidia.com/t/unable-to-create-plugin-for-batchednmsdynamic-trt-for-onnx-tao-model-using-graphsurgeon/300267/3 "2024-08-16T08:16:48Z")
**There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks** ---
**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\ **Post date:** [August 30, 2024, 8:17am UTC](https://forums.developer.nvidia.com/t/unable-to-create-plugin-for-batchednmsdynamic-trt-for-onnx-tao-model-using-graphsurgeon/300267/4 "2024-08-30T08:17:09Z")
Closed on August 30, 2024, 8:17am UTC This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.