Opencv with rtsp source and backgroundsubtractor in Xavier AGX

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
,
1

I have a doubt with opencv with cuda and the backgroundsubtractor in the Xavier AGX.

I’m using the next code:

int main(int argc, char** argv ) {
 VideoCapture cap(0);
   if(!cap.isOpened()) {
       return -1;
   }

     cv::Ptr<cv::cuda::BackgroundSubtractorMOG2> pBackSub = cv::cuda::createBackgroundSubtractorMOG2(5, 750, false);

   while(true) {
       Mat img;
       cap >> img;

       cv::cuda::GpuMat gpu;
       cv::cuda::GpuMat fgMask;
       gpu.upload(img);


   
     pBackSub->apply(gpu, fgMask, 0.2);

       Mat mask;
       fgMask.download(mask);


       imshow("image", mask);
       waitKey(12);
   }

 return 0;
}

With this test code the subtractor works fine, but when I change the videocapture to a rtsp source I start to see the mask with a lot of noise and the camera is stopped, so it should be a black image as I see in the example above.

Besides, if I test that code in other desktop pc I don’t have this problem.

Does anyone know what is happening?

Thanks in advance

3

Hi,
Please refer to the samples about launching RTSP source:
I got the RTSP video stream on the jetson nano and ran it for about 50 frames before returning false - #7 by DaneLLL
Doesn't work nvv4l2decoder for decoding RTSP in gstreamer + opencv - #3 by DaneLLL

See if you can run the samples and apply to your usecase.

4

With the BigBuckBunny_115k example I see good results, but I don’t understand what the problem is with my rtsp camera, do you have any idea what the problem is?

Thanks

5

Maybe the difference is resolution and pixels/s. The online sample has very small resolution, while probably your IP cam has much higher resolution and opencv videoio may be the bottleneck.
Also note that imshow is CPU only and not efficient on Jetsons. You may use a VideoWriter with a gstreamer pipeline to another display sink, or jetson-utils VideoOutput.

For this kind of GPU processing, you may try to use nvivafilter plugin after H264 decoding with frames in NVMM memory.
You may have a look to this example or this one for opencv GPU processing RGBA frames in NVMM memory (you may run the BG substractor on RGB only, not sure what it would do on alpha),
For NV12 processing being faster, this example may also help, but this might not be your case.

1 Like