Hi,
I’m using a Jetson Nano to (first phase) read video from multiple cameras and publish them on a web a site (probably will scale down resolutions and framerate).
The thing is that I’m trying to use nvsteammux because on the next phase I’ll try to add some inference and tracking steps.
Reading each camera and saving to a mp4 on different pipelines works well:
gst-launch-1.0 -e nvarguscamerasrc ! nvvidconv ! x264enc ! mp4mux ! filesink location=csi_cam.mp4 gst-launch-1.0 -e rtspsrc location="rtsp://user:password@192.168.200.12:554" drop-on-latency=1 ! rtph264depay ! decodebin ! nvvidconv ! x264enc ! mp4mux ! filesink location=rtsp_cam.mp4
But when I add a nvstreammux element (and after that a nvmultistreamtiler to mix them), the rtsp part of the inputs doesn’t generate anything and the result is just a video from the CSI camera. Also tested each input separately, for instance:
gst-launch-1.0 -e nvarguscamerasrc bufapi-version=1 maxperf=1 ! m.sink_0 nvstreammux name=m live-source=1 width=1920 height=1080 batch_size=1 ! nvmultistreamtiler width=2560 height=1920 ! nvvideoconvert ! x264enc ! mp4mux ! filesink location=csi_cam.mp4
does generate some video (framerate seems to drop down though), whereas:
gst-launch-1.0 -e rtspsrc location="rtsp://user:password@192.168.200.12:554" ! rtph264depay ! decodebin ! m.sink_0 nvstreammux name=m live-source=1 width=1920 height=1080 batch_size=1 batched-push-timeout=400000 ! nvmultistreamtiler width=2560 height=1920 ! nvvideoconvert ! x264enc ! mp4mux ! filesink location=rtsp_cam.mp4
does not.
When adding some debugging GST_DEBUG=decodebin:7, it reports that frames are being dropped by nvv4l2decoder:
0:00:00.536314530 22108 0x7f50004370 DEBUG decodebin gstdecodebin2.c:5588:gst_decode_bin_handle_message: Forwarding msg qos message: 0x7f40392910, time 99:99:99.999999999, seq-num 264, element 'nvv4l2decoder0', GstMessageQOS, live=(boolean)false, running-time=(guint64)317513471, stream-time=(guint64)233237794, timestamp=(guint64)317513471, duration=(guint64)18446744073709551615, jitter=(gint64)-317513472, proportion=(double)0.5, quality=(int)1000000, format=(GstFormat)GST_FORMAT_BUFFERS, processed=(guint64)1, dropped=(guint64)1; 0:00:00.556726784 22108 0x7f50004370 DEBUG decodebin gstdecodebin2.c:5588:gst_decode_bin_handle_message: Forwarding msg qos message: 0x7f40392990, time 99:99:99.999999999, seq-num 265, element 'nvv4l2decoder0', GstMessageQOS, live=(boolean)false, running-time=(guint64)350792162, stream-time=(guint64)266516485, timestamp=(guint64)350792162, duration=(guint64)18446744073709551615, jitter=(gint64)-350792163, proportion=(double)0.5, quality=(int)1000000, format=(GstFormat)GST_FORMAT_BUFFERS, processed=(guint64)1, dropped=(guint64)2; 0:00:00.596653045 22108 0x7f50004370 DEBUG decodebin gstdecodebin2.c:5588:gst_decode_bin_handle_message: Forwarding msg qos message: 0x7f40392a10, time 99:99:99.999999999, seq-num 268, element 'nvv4l2decoder0', GstMessageQOS, live=(boolean)false, running-time=(guint64)384053976, stream-time=(guint64)299778299, timestamp=(guint64)384053976, duration=(guint64)18446744073709551615, jitter=(gint64)-384053977, proportion=(double)0.5, quality=(int)1000000, format=(GstFormat)GST_FORMAT_BUFFERS, processed=(guint64)1, dropped=(guint64)3; 0:00:00.641135901 22108 0x7f50004370 DEBUG decodebin gstdecodebin2.c:5588:gst_decode_bin_handle_message: Forwarding msg qos message: 0x7f40392a90, time 99:99:99.999999999, seq-num 271, element 'nvv4l2decoder0', GstMessageQOS, live=(boolean)false, running-time=(guint64)417307768, stream-time=(guint64)333032091, timestamp=(guint64)417307768, duration=(guint64)18446744073709551615, jitter=(gint64)-417307769, proportion=(double)0.5, quality=(int)1000000, format=(GstFormat)GST_FORMAT_BUFFERS, processed=(guint64)1, dropped=(guint64)4;
I also implemented a very similar pipeline in C, and capturing QoS messages shows a very similar output:
Received message on bus: source nvv4l2decoder0, msg_type qos live: 0, running time: 2360412960, stream time: 1994349901, timestamp: 2360412960, duration: -1 format: 4, processed: 1, dropped: 53 Received message on bus: source nvv4l2decoder0, msg_type qos live: 0, running time: 2393746293, stream time: 2027683234, timestamp: 2393746293, duration: -1 format: 4, processed: 1, dropped: 54 Received message on bus: source nvv4l2decoder0, msg_type qos live: 0, running time: 2427079626, stream time: 2061016567, timestamp: 2427079626, duration: -1 format: 4, processed: 1, dropped: 55 Received message on bus: source nvv4l2decoder0, msg_type qos live: 0, running time: 2460412959, stream time: 2094349900, timestamp: 2460412959, duration: -1 format: 4, processed: 1, dropped: 56 Received message on bus: source nvv4l2decoder0, msg_type qos live: 0, running time: 2493746293, stream time: 2127683234, timestamp: 2493746293, duration: -1 format: 4, processed: 1, dropped: 57
I’m fairly new to gstreamer and GPUs and have not been able to find more information on this problem. Any advice is greatly appreciated.