Jetson Nano Save two usb webcam streams picture in picture pipeline wont start

Hi,

I am trying to capture 2 webcam streams and save them as picture in picture using gstreamer. I am using the videomixer in gstreamer. One stream is 1080p with audio and other is 360p no audio. Below is the pipeline.

gst-launch-1.0 -e v4l2src device=/dev/video1 ! queue ! 'video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1' ! queue ! mix. v4l2src device=/dev/video0 ! queue ! 'video/x-raw, framerate=(fraction)30/1' ! queue ! videomixer name=mix ! nvvidconv ! nvv4l2h264enc maxperf-enable=1 bitrate=4000000 profile=4 ! h264parse ! audiomix. pulsesrc device="alsa_input.usb-VXIS_Inc_ezcap_U3_capture-02.analog-stereo" ! 'audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=44100, channels=(int)2' ! queue ! audioconvert ! voaacenc ! aacparse ! mpegtsmux name=audiomix ! filesink location="recorded.ts"

The issue is when I run the above pipeline. it wont start. Gets stuck without starting.

Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstPulseSrcClock
Redistribute latency…

Nothing after this. What could be the reason for this?

Hi,
You may utilize nvcompositor. Please refer to

$ gst-launch-1.0 videotestsrc ! video/x-raw,format=UYVY,width=1920,height=1080 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_0  videotestsrc ! video/x-raw,format=UYVY ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! comp.sink_1 nvcompositor name=comp sink_1::xpos=10 sink_1::ypos=10 sink_2::xpos=50 sink_2::ypos=50 ! nvoverlaysink
1 Like

Hi Danelll,

Thank you for the reply. I have tried the below pipline

gst-launch-1.0 -e v4l2src device=/dev/video1 ! queue ! ‘video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1’ ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=RGBA’ ! comp.sink_0 v4l2src device=/dev/video0 ! queue ! ‘video/x-raw, framerate=(fraction)30/1’ ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=RGBA’ ! comp.sink_1 nvcompositor name=comp sink_1::xpos=10 sink_1::ypos=10 sink_2::xpos=0 sink_2::ypos=0 ! nvvidconv ! nvv4l2h264enc maxperf-enable=1 bitrate=4000000 profile=4 ! h264parse ! audiomix. pulsesrc device=“alsa_input.usb-VXIS_Inc_ezcap_U3_capture-02.analog-stereo” ! ‘audio/x-raw, format=(string)S16LE, layout=(string)interleaved, rate=44100, channels=(int)2’ ! queue ! audioconvert ! voaacenc ! aacparse ! mpegtsmux name=audiomix ! filesink location=“recorded.ts”

Now the videos can be captured without any problem picture in picture. But when audio is added the pipeline starts to give warnings and the process stops without exiting the pipeline. Is there any fix for this issue?

Hi,
A few suggestions:

  1. You may run audio-only pipeline to confirm audio source is good.
  2. Set sync=false in filesink. This can disable A/V sync mechanism of gstreamer
  3. Run sudo jetson_clocks.

One case we have tried is onboard camera + USB audio input. FYR.

1 Like

Thank you Danelll. I will test this method.