Run inference "on demand"

How can I run inference “on demand” from the code (when some requirement is met)? I can’t find any example of this. Also, how to only run inference on some streams (not on all the streams in the batch)?

Even if it was something like disabling inference by attaching something to the frame meta. I’ve seen that should_infer_object inside gstnvinfer.cpp looks at rect_params. I can obviously modify those to make sure it doesn’t run the inference, but that seems very hacky and it only works for objects already detected (won’t work for primary inference).

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Is it possible add one property “enable-infer” in nvinfer?

Yes, adding an enable-infer per NvDsFrameMeta that is checked by nvinfer would solve a lot of my issues. This means I’m not forced to run specific models on streams that don’t need it (very useful for embedded solutions). Currently I’m forced to run the models on all streams and ignore the result which wastes resources.

Sorry for later response, Do you fixed your issue?

The only hack I’ve come up with is to change the interval on the fly. Set it to 0 when I want to run it, and set it to INT_MAX when I don’t want it to run.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.