Rtpjitterbuffer in gstreamer pipelines

Autonomous Machines Jetson & Embedded Systems Jetson TX2
,
1

I’m currently tuning a pair of gstreamer pipelines involving a Jetson TX2 sending and receiving audio and video. I’m trying to track down what’s causing graphics artifacts on the stream coming from the TX2 where a camera streaming h264 (natively generating the h264 stream, the TX2 is only doing the payloading and sending) is capturing and streaming via rtp. Performance on this transmission stream is acceptable until the video receive pipeline is brought on line, at that point the transmission stream’s video quality drops precipitously.

I’ve set up a dynamic gstreamer pipeline on the receiving desktop that logs debug statements and utilizes a rtpjitterbuffer with 0 latency to monitor stats on the stream the TX2 is transmitting.

Conventional gstreamer wisdom is that adding a queue and/or a rtpjitterbuffer to the receiving desktop pipeline should ameliorate some of the issues. Adding and removing queues and jitterbuffers with various settings results in an average of about 100 packets lost per 100k packets pushed over 12 different tests. Needless to say video quality doesn’t improve since without queues or jitterbuffers the average lost packets is 95 per 100k.

Now, in defiance of conventional wisdom and documentation, I’ve also tested placing queues and jitterbuffers on the TX2 transmission side pipeline and discovered dramatic improvement. The packet loss on the receiving side is cut in half by the simple addition of a queue (52 per 100k), and when combined with a jitterbuffer (even with the latency set to 0) the packet loss drops to 0 per 100k.

The question is: Why? While it’s great having a solution that completely mitigates the issue I’m seeing, I also need an plausible explanation to bolster support for this solution. Does anybody have ideas? Perhaps, places to look on the TX2 side for evidence?
(also posted on stack overflow)

2

Not sure at all, but you may try increasing TX sockets buffer max size:

sudo sysctl -w net.core.wmem_max=33554432
sudo sysctl -w net.core.wmem_default=33554432
3

Thanks @Honey_Patouceul, this was good suggestion.

After digging quite a bit more it looks as though this problem revolves around simultaneously sending and receiving multi-media. In our system the TX2’s job is to send and receive video/audio, packet loss is occurring when the encoder sending AV to the TX2 is active on the network (it’s multi-casting). Reading through gstreamer debugging and wireshark logs tells the tale of packets being lost when both the AV encoder and the TX2 are bursting packets related to IDR frames (video key frames). Setting up wireshark on both ends of the Jetson’s video stream shows that the missing packets are making it to the interface driver but disappear by the time they reach the recipient computer’s interface. It’s tempting to just write this off as network loss, however, the network is not busy and looking at the I/O traffic on the interface card reveals that it’s nowhere near capacity (about 11 Mbit of a 100 Mbit connection).

Still trying to figure out why a Tx side jitterbuffer fixes this packet loss problem. Per suggestion I’ve increased the size of the write and read buffers on the network interface kernel settings, this did not remedy the problem. Checking with netstat -sanu does show some ‘send buffer errors’ being registered with everything running, but network monitoring with applications like bmon do not report lost packets. Currently looking to recompile the kernel so the Jetson will let me run dropwatch.