Overview
I am trying to stream compressed video from a Xavier via RTP over UDP to a remote PC.
I require the stream to be h264 or h265 (since we will be using the hardware encoders), and will be transported over an isolated ethernet local network.
I am having issues with the udpsink on the xavier NX and AGX devices. It looks like it is sending an empty payload.
My questions are:
- Is this a known issue? If not, is it reproducible?
- If so, is there a workaround that doesn’t involve setting up a RTSP server?
- If not, what are my next steps to trying to diagnose this issue.
Problem Description
On the Host PC (Windows running Ubuntu 18.04 on WSL1, address 192.168.1.207 with all firewalls turned off, gstreamer 1.14.5 installed via apt) the following pipeline works
gst-launch-1.0 videotestsrc ! video/x-raw,width=1280,height=720,framerate=30/1,format=I420 ! x265enc speed-preset=superfast tune=zerolatency ! h265parse config-interval=1 ! rtph265pay name=pay0 pt=96 ! udpsink port=5000 host=192.168.1.207 buffer-size=1024 blocksize=512
in the sense that the following SDP
c=IN IP4 192.168.1.207
m=video 5000 RTP/AVP 96
a=rtpmap:96 H265/90000
is read by VLC, and displayed on the screen.
On either Xavier, running Jetpack 4.4 (AGX) or 4.5.1 (NX) running exactly the same pipeline
with the exactly the same SDP (except with the IP addresses changed to the corresponding devices) does not work in the sense that VLC does discover the stream but reports hevc info: Waiting for VPS/SPS/PPS.
Some Testing
Initially, i thought this was an issues with VLC. So instead I used gstreamer as the client.
In particular i used the following client pipeline
gst-launch-1.0 udpsrc port=5000 ! 'application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H265,payload=(int)96' ! rtph265depay ! h265parse ! matroskamux ! filesink location=test.mkv.
I performed the following test on different devices:
- On the server: run the first (udpsink) pipeline with the IP address pointed at the correct destination.
- On the client: run the second (file recorder) pipeline.
- After a couple of seconds, copy (if necessary) the file to the windows PC and play it in VLC.
The results are:
- Server(PC) → Client(PC) works
- Server(PC) → Client(AGX) works
- Server(AGX) → Client (PC) fails
- Server(AGX) → Client(AGX) fails
Additionally, running listening (with netcat) to the udp port, shows that udp packets are being transmitted, but it looks like the payload is more-or-less empty.
Finally, to make sure i didn’t have issues with payloading and parsing i ran
gst-launch-1.0 videotestsrc ! 'video/x-raw,width=1280,height=720,format=I420,framerate=30/1' ! x265enc ! h265parse config-interval=-1 ! rtph265pay pt=96 ! 'application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H265,payload=(int)96' ! rtph265depay ! h265parse ! matroskamux ! filesink location=test.mkv on the AGX, which successfully recorded videotestsrc to test.mkv and was playable in VLC.
Debug info
I’ve attached the level-3 gstreamer debug file for reference.
debug.txt (81.0 KB)
In Closing
An idea solution for us would be to get the UDP sink working. Any help with this would be much appreciated.
We’re tied to Jetpack 4.4 on the AGX due to driver dependencies.
We are already exploring alternatives using RTSP, but that introduces overhead on other parts of our software stack, and we would prefer if this was not necessary.
Many Thanks.
Pete. C
(edit: attached debug.txt)