Segmentation fault in the pipeline having tcpserversink with multiple clients

Hello!

I’m experimenting with transmitting the video processed on Jetson Nano to several clients in the local network using certain GStreamer pipeline (please, see below) and have suddenly found, that it crashes quite regularly.

Here’s a minimal GStreamer pipeline (server part), which can be used to reproduce the crash:

gst-launch-1.0 nvarguscamerasrc sensor-mode=3 bufapi-version=1 ! queue ! videoconvert ! nvv4l2vp8enc bufapi-version=1 ! gdppay ! tcpserversink host=0.0.0.0 port=4455

And here’s a client pipeline:

gst-launch-1.0 tcpclientsrc host=jetson.local port=4455 ! fakesink

Note the sensor-mode parameter of nvarguscamerasrc and nvv4l2vp8enc encoding element in the server pipeline, this is the exact configuration, which the crash can be reproduced with. I usually start 2-3 of client pipelines (on other machines in the network) and in 4-5 minutes here’s the following on the Jetson Nano:

Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 7 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 7 
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21,000000 fps Duration = 47619048 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28,000001 fps Duration = 35714284 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59,999999 fps Duration = 16666667 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120,000005 fps Duration = 8333333 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 3 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 59,999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

(gst-launch-1.0:15183): GLib-GObject-WARNING **: 21:50:45.343: ../../../../gobject/gsignal.c:2641: instance '0x559cf65c40' has no handler with id '43555'
    Caught SIGSEGV
#0  0x0000007f9aaeb038 in __GI___poll (fds=0x559cf82a00, nfds=548057092664, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/poll.c:41
#1  0x0000007f9abf8e08 in  () at /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0
#2  0x000000559ccec2e0 in  ()
Spinning.  Please run 'gdb gst-launch-1.0 15183' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

I’m using the latest JetPack image with all the updates. Again, neither with other sensor modes, nor with other encoders the crash can be reproduced for me, only in this exact configuration.

Would appreciate any help with this. Thanks in advance!

Forgot to mention a topic with quite similar result, but different elements used: streaming using jpegenc halts after a short delay.

Hi,
You may replace videoconvert with nvvideoconvert and try again. Need to send video/x-raw(memory:NVMM) buffers to encoder.

Hi, @DaneLLL, thanks for your suggestion! Still, I have to confirm, that it crashes with nvvideoconvert element instead of videoconvert as well.

From my recent experiments, I can also confirm, that the crash is reproducible with nvv4l2h264enc and omxvp8enc encoder elements, when placed instead of nvv4l2vp8enc element in the pipeline mentioned above.

But I can also confirm, that it doesn’t crash for me even in long term experiment with other sensor modes, e.g. sensor-mode=4.

As per my recent experiment, omxh264enc is also affected with the crash.

Hi,
Not sure but it may be specific to TCP streaming. Please try UDP streaming. Here is a reference post:

Would need to replace v4l2src ! nvvidconv with nvarguscamerasrc.

Thanks for the suggestion, @DaneLLL, indeed I haven’t tried it with UDP streaming. But the problem with my exact configuration is that, to the best of my knowledge, udpsink doesn’t support multiple unicast connections, i.e. it’s not server itself, so I have to resort to tcpserversink for this functionality. And in case of single connected client this segfault never happens with tcpserversink as well. Anyway, thanks for pointing out to this partial workaround!

Hi,
Mode 4 is 720p120 and could you try to run the mode in 60fps? The command should be

gst-launch-1.0 nvarguscamerasrc sensor-mode=4 bufapi-version=1 ! 'video/x-raw(memory:NVMM),width=1280,height=720,framerate=60/1' ! queue ! nvvideoconvert ! nvv4l2vp8enc bufapi-version=1 ! gdppay ! tcpserversink host=0.0.0.0 port=4455

Hi, @DaneLLL,

Have just tried the exact command you’ve proposed with 3 connected clients, unfortunately, it fails the same way:

(gst-launch-1.0:7292): GLib-GObject-WARNING **: 11:31:53.496: ../../../../gobject/gsignal.c:2641: instance   '0x55944006b0' has no handler with id '19564'
Caught SIGSEGV
#0  0x0000007fa84e5038 in __GI___poll (fds=0x55943c6b60, nfds=548285657656, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/poll.c:41
#1  0x0000007fa85f2e08 in  () at /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0
#2  0x00000055941613a0 in  ()

Hi,
So it works in 720p120 and fails in 720p60?

Well, I have to correct myself — I’ve just double checked with this command

 gst-launch-1.0 nvarguscamerasrc sensor-mode=4 bufapi-version=1 ! queue ! nvvideoconvert ! nvv4l2vp8enc bufapi-version=1 ! gdppay ! tcpserversink host=0.0.0.0 port=4455

which is 720p120 and it fails as well. So probably it has nothing to do with camera sensor-modes or codecs used, as I used to think, only the running time before segmentation fault fluctuates.