Output video did not show up when running c++ detectnet on a video with soundtrack

Hello everyone,

We have built jetson-inference on two Jetson-Nano devices, with one was built from source and the other one use the docker container.

When running the c++ version of the detectnet, we noticed that there’s a difference regarding output video being displayed or not.

Case 1: Inference on a video captured from a webcam on Microsoft Windows which has soundtrack

  • command: detectnet input_video_webcam.mp4 output_video.mp4
  • result: output video did not show up during the inference
  • the error message:
    Opening the glx window fails within “glDisplay.cpp” at the following line of code:
    “GLXFBConfig* fbConfig = glxChooseFBConfig(mDisplayX, screenIdx, fbAttribs, &fbCount);”
    fbCount = 0 and fbConfig = NULL after the call.

Case 2: Inference on a video captured using Jetson Nano with opencv from CSI camera IMX477 without soundtrack

  • command: detectnet input_video_csi.mp4 output_video.mp4
  • result: output video did show up during the inference

However when running the python version detecnet.py on both the webcam and also csi videos, the output did show up during inference.

All videos are in mp4 format. Also scaling down the webcam video from 1920x1280 to 960x640 pixels and reducing the frame rate to 10fps using ffmpeg doesn’t solve the problem.

The webcam video can be played with both “video-viewer” and “video-viewer.py” without any error.

Then we tried this inference on another Jetson Nano that runs “detectnet” inside the docker container (file dates are may the third 2023):

  • the output did show up during inference on the webcam video

Removing the audio track from the webcam video and using detectnet afterwards will show the output during detection.

So my questions are:

  • Why does running c++ version of detectnet inside a docker container on a video with sound work as expected, but running detectnet built from source won’t show the output video during inference?
  • Why does removing the audio track solve the problem for detectnet built from source?
  • Why does detecnet.py display the output video during inference.

I looked into the program code of “detectnet.cpp” and “detectnet.py”.
The interesting thing is, that python and C code ends up at the same place “videoOutput:::Create()” with similar parameters
within the same shared lib “libinference-utils.so” but one works and the other one does not.

Next i commented part for part out of “detectnet.cpp”.
When this line is commented “detectNet::Create()”, the live video shows up. (line 108)
If the line: “detectNet* net = detectNet::Create(cmdLine);” is executed, the live video fails to show up.

(Sometimes, this look like a big memory overflow problem somewhere … )

Hopefully, someone has an idea.
Thanks.

=== system information from jtop ===

Ubuntu 18.04 Bionic Beaver
Release: 4.9.337-tegra
Python: 3.6.9
CUDA: 10.2.300
cuDNN: 8.2.1.32
TensorRT: 8.2.1.9
Vulkan: 1.2.70
OpenCV: 4.1.1 without CUDA
Module: Nvidia Jetson nano module (16GB eMMC)
SoC: tegra210
L4T: 32.7.4
Jetpack: 4.6.4

Hi,

Would you mind sharing the " input_video_webcam.mp4" video with us?
So we can reproduce this internally and check it further.

Thanks.

Hi @anasri89, I’m unsure why that might be, maybe GL/GLX bindings changed by other libraries? But given your case, can you try creating the videoOutput interface before the videoInput interface?

Thank for your reply. I have attached two videos, one with audio and the other without audio. We have also tried with other videos and the same problem occurs.

Looking forward to your reply.

Thanks.
nvidia_forum.zip (23.1 MB)

p/s: Dusty, I will give your suggestion a try and report back to you.

Hallo dusty_nv,

we tried your suggestion to create videoOutput interface before videoInput interface. By doing this the display window did show up. But we don’t understand why. Maybe you can explain?

Thanks.

@anasri89 sorry, I’m not exactly sure except that some GStreamer codecs/plugins seem to mess with OpenGL, and only in obscure circumstances does this occur (I have only heard of it before with RTSP, and not been able to reproduce it myself). I’m not sure what triggers it, but in this case it seems to be something that GStreamer elements doing the MP4 decoding do.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.