How can we conditionally active or deactivate the secondary inference model.
Hello I am new to deepstream i am trying to run the test2 app with the pipeline of
source, h264parser, decoder, streammux, pgie, nvtracker, sgie1, sgie2, sgie3, nvvidconv,custom_plugin, nvosd, sink, I am trying to run sgie1 on condition like if tracker id == something or frameid = something then activate the sgie1 but I don’t know where should I start from should I
1.Write a custom plugin
2.Modify tracker code
3.or there is another method.
If the answer is a custom plugin then what should I follow?
If the answer is tracker code what file should I edit?
After long research, maybe I have found that gstnvinfer.cpp has what I need on line number 1439 the function should_infer_object here I added few lines its check the frame number if the frame number is greater than 500 then it returns false. But how to conform this logic is working? anyone know what I did is correct?
static inline gboolean
should_infer_object (GstNvInfer * nvinfer, GstBuffer * inbuf,
NvDsObjectMeta * obj_meta, gulong frame_num,
GstNvInferObjectHistory * history)
{
if (frame_num >500){
// g_printerr("%d",frame_num);
return FALSE;
}
if (nvinfer->operate_on_gie_id > -1 &&
obj_meta->unique_component_id != nvinfer->operate_on_gie_id)
return FALSE;
if (obj_meta->rect_params.width < nvinfer->min_input_object_width)
return FALSE;
if (obj_meta->rect_params.height < nvinfer->min_input_object_height)
return FALSE;
if (nvinfer->max_input_object_width > 0 &&
obj_meta->rect_params.width > nvinfer->max_input_object_width)
return FALSE;
if (nvinfer->max_input_object_height > 0 &&
obj_meta->rect_params.height > nvinfer->max_input_object_height)
return FALSE;
/* Infer on object if the operate_on_class_ids list is empty or if
* the flag at index class_id is TRUE. */
if (!nvinfer->operate_on_class_ids->empty () &&
((int) nvinfer->operate_on_class_ids->size () <= obj_meta->class_id ||
nvinfer->operate_on_class_ids->at (obj_meta->class_id) == FALSE)) {
return FALSE;
}
/* History is irrevelavant for detectors. */
if (history && IS_CLASSIFIER_INSTANCE (nvinfer)) {
gboolean should_reinfer = FALSE;
/* Do not reinfer if the object area has not grown by the reinference area
* threshold and reinfer interval criteria is not met. */
if ((history->last_inferred_coords.width *
history->last_inferred_coords.height * (1 +
REINFER_AREA_THRESHOLD)) <
(obj_meta->rect_params.width * obj_meta->rect_params.height))
should_reinfer = TRUE;
if (frame_num - history->last_inferred_frame_num >
nvinfer->secondary_reinfer_interval)
should_reinfer = TRUE;
return should_reinfer;
}
return TRUE;
}
There is no frame number counting inside nvinfer.
A possible solution is to implement in application level. It is easier for the application to count frame numbers. You can change the object meta from pgie to not meet the sgie limitation(e.g. change the unique_component_id to a strange number) when the frame count is larger than 500.
@Fiona.Chen Thanks for the reply I will try that method also and I have take frame_num as an example here but what I am trying to do is check how many times did secondary model ran on a specific primary object. If it had run 5 times then do not do secondary inference on that object. So is there any object or variable that keeps count of how many time secondary model ran or we should create our own variables?
The nvtracker may help such scenario. Gst-nvtracker — DeepStream 6.1.1 Release documentation
nvtracker can tracking the same object and sgie will skip inferencing on the same object.
You can find the nvtracker usage in deepstream-test2-app sample.C/C++ Sample Apps Source Details — DeepStream 6.1.1 Release documentation
Where can I find the tracker main code I have seen nvdstracker.h and nvbufsurface.h is there any file for example gst-infer has gstnvinfer.cpp
nvdstracker is proprietary plugin, it is not open source.
Ok but does nvdstracker has an option that the keeps track or count of how many time it has gone for secondary inference. If not how can I do that without nvdstracker code?
No. nvdstracker do not provide such settings. You need to customize your own tracking algorithm. Gst-nvtracker — DeepStream 6.1.1 Release documentation
@Fiona.Chen Hello I tried to Change the unique_component_id to some number but its not working when I checked nvinfer and print the value of unique_component_id the output is 1. But expected output should be 100 right?
This is in main app
if (frame_number > 500)
{
obj_meta->unique_component_id = 100;
g_print("**********%d\n", obj_meta->unique_component_id);
/* code */
}
But in nvinfer its not updating this value here output is 1
This is nvinfer
should_infer_object(GstNvInfer *nvinfer, GstBuffer *inbuf,
NvDsObjectMeta *obj_meta, gulong frame_num,
GstNvInferObjectHistory *history)
{
g_print("**********************%d", obj_meta->unique_component_id);
if (nvinfer->operate_on_gie_id > -1 &&
obj_meta->unique_component_id != nvinfer->operate_on_gie_id)
{
return FALSE;
}
Any idea why the var is not updating?
Have you put the code before sgie and after tracker?
I have used test2 app in sink_prob
No. This is the wrong place. You want the sgie to get this information so you can not change this after sgie. Please study basic gstreamer knowledge and programming skills ( GStreamer: open source multimedia framework) if you want to customize your own functions.