How to use INMSLayer with DeepStream 8.0 when dynamic sizes are not supported?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

I’m using DeepStream 8.0 (TensorRT 10.9) and need to add NMS to my ONNX model for object detection. Related to this ticket.

The issue:

  • DeepStream’s nvinfer plugin requires fixed output shapes (apart from batch sizes)
  • INMSLayer outputs dynamic shapes: selectedIndices[NumOutputBoxes, 3] as NumOutputBoxes is variable
  • EfficientNMS_TRT plugin outputs fixed padded shapes [batch, max_output_boxes, 4] which works with DeepStream 8.0.

According to TensorRT 10.12 release notes, EfficientNMS_TRT is now deprecated and INMSLayer should be used instead. However, INMSLayer’s dynamic outputs cause DeepStream to fail during engine initialization.

It’s not yet an issue as DS 8.0 uses an older version of TRT, where the plugin is not yet deprecated. However I would prefer to use the native ONNX layer which isnt due to be deprecated.

Questions:

  1. Is there a way to configure INMSLayer to output fixed padded shapes like EfficientNMS_TRT?
  2. Should I continue using the soon to be deprecated EfficientNMS_TRT for DeepStream compatibility?

Any help would be appreciated!

3

As the suggestion of TensorRT/plugin/efficientNMSPlugin/efficientNMSPlugin.cpp at release/10.12 · NVIDIA/TensorRT, the INetworkDefinition::addNMS() interface can be used to add the NMS layer to network.

TensorRT: nvinfer1::INetworkDefinition Class Reference

The " maxOutputBoxesPerClass" is similar to the max_output_boxes of EfficientNMS_TRT

4

Thanks for the reply.

Sorry I’m not following, how is it possible to access/change the INetworkDefinition when not using TensorRT directly?

My understanding is Deepstream abstracts that away, making it not possible to call those functions.

Thanks,
Magnus

5

The TensorRT is used in gst-nvinfer to parse model and generate model engine. Why do you need to change the interface?

6

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.