Saving Video File With Detection Boxes

I have been using Gstreamer to capture video from a USB camera and then using deepstream plugins to perform inference on each frame using the following pipeline:

v4l2src -> nvvidconv -> nvvidconv -> nvvideoconvert -> nvstreammux -> nvinfer -> appsink

With this I am able to get detection boxes and have my program act on them. Now to find out how to improve accuracy i wanted to also have the detected boxes be drawn on the image and saved to file. I am using files instead of a window as the Jetson is run headless. The Pipeline that i tried is as follows:

v4l2src -> nvvidconv -> nvvidconv -> nvvideoconvert -> nvstreammux -> nvinfer -> nvvideoconvert -> nvdsosd ->
nvvideoconvert -> nvv4l2h264enc -> h264parse -> qtmux -> filesink

I am skipping a lot of the capsfilters to keep things simple. When run this does generate a mp4 file however I am unable to read anything. Specifically when opening the file with Linux Videos and inspecting the properties I see that it has an Audio Property but is missing Video Properties. Does anyone know what might be the cause? Or if the pipeline should would work at all?

Another pipeline i have considered instead was to add a tee before the nvstreammux or earlier and saving the video to file in the fork and write the detection boxes to file in the other. Then use another program with opencv to draw on the video. Though I would prefer if the first option could work.

How are you closing your program? You need to send an End of Stream Signal before closing the pipeline so that it writes the headers to the mp4 file. If you’re not doing that, it may be the reason for why you’re unable to read your mp4 file.

1 Like

Currently I am closing the pipeline by setting the state to null. When sending the signal which element requires the signal?

I believe it is the filesink.

Thanks, will check it out and see if that works.

Got it to work using the EOS signal. Thanks for the help!