Implementation of timer callback function inside deepstream plugin

I need to implement a function with long processing inside deepstream plugin (gstdsexample) .
Not to block the pipeline’s fast processing, it is implemented outside the pipeline in asynchronous way using timer and std::mutex.

Timer is called every regularly time interval, get data with thread safe protection using std::mutex and do extra processing inside the function.

That function needs to have access

dsexample pointer
(GstDsExample *dsexample = GST_DSEXAMPLE (object);)

so that I can access some member data inside struct _GstDsExample{}.

My timer and callback function are as follow. Now this line dsexample->c_infos[cnt].copy(oneCamgridInfos); has error in compilation.

static void timer_start(std::function<void(void)> func, unsigned int interval)
{
  std::thread([func, interval]()
  { 
    while (true)
    { 
      auto x = std::chrono::steady_clock::now() + std::chrono::milliseconds(interval);
      func();
      std::this_thread::sleep_until(x);
    }
  }).detach();
}

static void interfacetoBackend()
{
  int oneCamgridInfos[NUMAVERAGE][ALL];
  int average[ALL];
  for(int cnt=0; cnt < NUMCAMERAS; cnt++){
     dsexample->c_infos[cnt].copy(oneCamgridInfos);
     std::cout << "Averaged of Camera " << cnt << std::endl;
     for(int ver=0; ver < ALL; ver++){
       average[ver]=0;
       for(int hor=0; hor < NUMAVERAGE; hor++){
          average[ver]+=oneCamgridInfos[ver][hor];
       }
       std::cout << average[ver] << " ";
     }
     std::cout << std::endl;
  }
  
}

My code is attached.
gstdsexample.cpp (36.3 KB) gstdsexample.txt (4.9 KB)

How can I have access this pointer dsexample-> inside my callback function?

I don’t understand where your “dsexample->” comes from and how to use it.
Glib already have timer call “g_timeout_add()”, you don’t need to write your own timer call.

static void interfacetoBackend()
is timer callback function and to access data member inside struct _GstDsExample{} , I need dsexample pointer. Isn’t it? Like in the following function.

static gboolean
gst_dsexample_start (GstBaseTransform * btrans)
{
    GstDsExample *dsexample = GST_DSEXAMPLE (btrans);
    .......................
    dsexample->batch_size;
   ............................
} 

So how to have access dsexample pointer in user addin function?

I’ll look into g_timeout_add(). Thanks.

Gstreamer plugin is designed to handle input and output in the pipeline. It is no use if the “long processing” thread is inside the plugin. It is not recommended to do.

That is why I put in timer and put a separate thread and run “long processing” inside that separate thread. Is it what you suggested before as asynchronous process not to block the pipeline?

That depends on how you handle “dsexample->” in your call back, most element instances inside “dsexample->” will impact the thread synchronization, it will also block the pipeline.

Plugin is not a thread or process concept. It is a kind of interface.

So if I don’t want to use “dsexample->” in callback, how to share data from the pipeline? May I have any suggestion? What could be better apprpach?

I need to access bounding boxes in plugin (i have probe in plugin), then do homographic and send to backend server.

Doing homographic and sending to backend server takes time, that is why I implemented asynchronous in timer and callback in separate thread. But need to share data inside probe function.

How can I implement it this case?

It can be done in application level. In most of the sample codes there are samples of pad probe and the object meta is got inside the probe function.(deepstream-test1, deepstream-test2, deepstream-test3,…)

There is detector_bbox_info inside object meta which can provide coordinations of bbox.
https://docs.nvidia.com/metropolis/deepstream/dev-guide/DeepStream_Development_Guide/baggage/struct__NvDsObjectMeta.html

Hi,
For sending data to server, you may also refer to deepstream-test4 and deepstream-test5 samples.

Yes I can access bounding box in probe.
And put the boxes inside the array.
The array is inside struct _GstDsExample{} this Structure.

I need to process that array in callback function asynchronously. So that I don’t block the pipeline.
To access array inside struct _GstDsExample{}, without this pointer

GstDsExample *dsexample = GST_DSEXAMPLE (object);

How to access?

If you want to decouple the pipeline and you sending thread, the array should not be inside struct _GstDsExample{}. For example, copy it out to a global array which can be accessed both by probe function and you transformation thread. Don’t forget to do the thread synchronization with the global array

Ic ic thanks, let me test with global array. Thank you.

Don’t forget to do the thread synchronization with the global array

Sure thanks

Yes @DaneLLL. Those are kafka, azure, etc.
My backend uses IIS webserver and Gsoap.
Deepstream accpets custom messaging plugin. It needs me taking time to develop.
I am thinking to use URL to send data to backend at first since I need to show demo soon.

I’ll implement homographic and sending data back to backend server with URL inside a separate thread not to block the pipeline.

I have tested interfacing to IIS webserver from deepstream sample using URL, it works. So I need to implement not to block the pipeline.

We already have samples of sending specific msg to cloud server with nvbroker, you can refer to the sample of deepstream-test4 and deepstream-test5. You may refer to them.

Thanks. I used global array and can process in thread without blocking pipeline.

1 Like