Gstreamer video recording randomly stops using multiqueue and four cameras

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

I’m trying to debug a problem on my Xavier and I’m not quite sure where to start.

I have the e-consystems four camera system for Xavier. All of my cameras work great. But with my current video recording pipeline, recording just seems to stop sometimes after only a handful of seconds. I start my script and four video files are created. The video files grow in size for a bit, and then they just stop growing. There’s no change in output in the script and verbose mode didn’t provide anything helpful. I’ve tried a bunch of different combinations with the multiqueue buffer options but nothing helped. Sometimes longer recordings work fine, but often it just stops as described.

Here is my script. Please let me know if it looks okay? How could I debug this? Thanks!

sudo nvpmodel -m 0
sudo jetson_clocks


# WIDTH=1280
# HEIGHT=720
# EXPOSURE=105
# BITRATE=40000000

# WIDTH=4192
# HEIGHT=3120
# EXPOSURE=205
# BITRATE=40000000

WIDTH=3840
HEIGHT=2160
EXPOSURE=6
BITRATE=80000000

for i in 0 1 2 3
do
   v4l2-ctl -d $i --set-ctrl=frame_sync=1
   #v4l2-ctl -d $i --set-ctrl=auto_exposure=0
   #v4l2-ctl -d $i --set-ctrl=auto_exposure=1
   v4l2-ctl -d $i --set-ctrl=exposure_time_absolute=$EXPOSURE
done

sleep 2

filename=$(date +"%m_%d_%Y_%H:%M:%S")
gst-launch-1.0 -e \
multiqueue sync-by-running-time=TRUE max-size-bytes=0 max-size-buffers=0 max-size-time=0 name=mqueue \
v4l2src device=/dev/video0 ! "video/x-raw, format=(string)UYVY, width=(int)$WIDTH,height=(int)$HEIGHT" !  mqueue.sink_1 \
v4l2src device=/dev/video1 ! "video/x-raw, format=(string)UYVY, width=(int)$WIDTH,height=(int)$HEIGHT" !  mqueue.sink_2 \
v4l2src device=/dev/video2 ! "video/x-raw, format=(string)UYVY, width=(int)$WIDTH,height=(int)$HEIGHT" !  mqueue.sink_3 \
v4l2src device=/dev/video3 ! "video/x-raw, format=(string)UYVY, width=(int)$WIDTH,height=(int)$HEIGHT" !  mqueue.sink_4 \
mqueue.src_1 ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvv4l2h265enc  bitrate=$BITRATE ! h265parse ! qtmux ! filesink location=/media/external/Videos/{$filename}_0.mp4 \
mqueue.src_2 ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvv4l2h265enc  bitrate=$BITRATE ! h265parse ! qtmux ! filesink location=/media/external/Videos/{$filename}_1.mp4 \
mqueue.src_3 ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvv4l2h265enc  bitrate=$BITRATE ! h265parse ! qtmux ! filesink location=/media/external/Videos/{$filename}_2.mp4 \
mqueue.src_4 ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvv4l2h265enc  bitrate=$BITRATE ! h265parse ! qtmux ! filesink location=/media/external/Videos/{$filename}_3.mp4
3

Hi,
You should not need multiqueue. The pipeline can be run like:

gst-launch-1.0 v4l2src device=/dev/video0 ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=out_0.mp4 v4l2src device=/dev/video1 ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=out_1.mp4 device=/dev/video2 ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=out_2.mp4 device=/dev/video3 ! nvvidconv ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=out_3.mp4

Please give it a try.

4

Thank you! I suppose I don’t know how I ended up with the multiqueue. Your line did give me an error, but I found other examples and was able to update my script. This works great:

sudo nvpmodel -m 0
sudo jetson_clocks


# WIDTH=1280
# HEIGHT=720
# EXPOSURE=105
# BITRATE=40000000

# WIDTH=4192
# HEIGHT=3120
# EXPOSURE=205
# BITRATE=40000000

WIDTH=3840
HEIGHT=2160
EXPOSURE=6
BITRATE=80000000

for i in 0 1 2 3
do
   v4l2-ctl -d $i --set-ctrl=frame_sync=1
   #v4l2-ctl -d $i --set-ctrl=auto_exposure=0
   #v4l2-ctl -d $i --set-ctrl=auto_exposure=1
   v4l2-ctl -d $i --set-ctrl=exposure_time_absolute=$EXPOSURE
done

sleep 2

filename=$(date +"%m_%d_%Y_%H:%M:%S")

gst-launch-1.0 -e \
v4l2src device=/dev/video0 ! "video/x-raw, format=(string)UYVY, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvv4l2h265enc  bitrate=$BITRATE ! h265parse ! qtmux ! filesink location=/media/external/Videos/{$filename}_0.mp4 -e \
v4l2src device=/dev/video1 ! "video/x-raw, format=(string)UYVY, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvv4l2h265enc  bitrate=$BITRATE ! h265parse ! qtmux ! filesink location=/media/external/Videos/{$filename}_1.mp4 -e \
v4l2src device=/dev/video2 ! "video/x-raw, format=(string)UYVY, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvv4l2h265enc  bitrate=$BITRATE ! h265parse ! qtmux ! filesink location=/media/external/Videos/{$filename}_2.mp4 -e \
v4l2src device=/dev/video3 ! "video/x-raw, format=(string)UYVY, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)I420, width=(int)$WIDTH,height=(int)$HEIGHT" ! nvv4l2h265enc  bitrate=$BITRATE ! h265parse ! qtmux ! filesink location=/media/external/Videos/{$filename}_3.mp4 -e \

Thanks for helping me get to the bottom of this!