Good evening to everyone!
I apologize in advance if this might be the wrong section to post the issue I am facing. I am an aerospace engineering with little experience in programming (mainly Matlab and Python) and this is my first post, I hope to express myself clearly enough.
Basically, for a reserach project I need to perform some computer vision analysis from a video stream from the raspberry HQ cam (IMX477 sensor) on the Jetson Xavier AGX using a Simulink model that I have successfully developed, compiled and deployed on the board. The output of the computer vision algorithms is sent back to the host pc where another simulink model is running (through UDP) to be used as input for further calculations. Since the Jetson Xavier AGX doesn’t have a CSI input port and since we have available a Jetson Nano, I have connected the camera to the jetson Nano, and I am trying to stream the video using Gstreamer from Nano to Xavier via UDP/RTP encoded in h264/MPEG4 with the highest framerate as possible and the lowest latency. I have tried this pipeline with excellent results between the jetson Nano (server) and my host pc running windows 10 (client).
Jetson nano (server):
gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)60/1’ ! nvvidconv flip-method=vertical-flip interpolation-method=Bilinear ! ‘video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)NV12’ ! nvv4l2h264enc insert-sps-pps=true maxperf-enable=true idrinterval=15 poc-type=2 ! h264parse ! rtph264pay pt=96 ! udpsink host=188.8.131.52 port=5000 sync=false -e
Windows 10 workstation (client)
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink
The pipeline works flawlessly, with close to zero latency and (at least judging by eye, seems like the stream is handled indeed at 60 fps because it is very fluid on my laptop screen, which has 60HZ of refresh rate).
The problem arises when I am trying to stream the video to the Jetson Xavier instead of the windows workstation. Using the same client pipeline, I get this error:
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
#0 0x0000007f8ab1aef8 in __GI___poll (fds=0x5560457360, nfds=547788850056, timeout=) at …/sysdeps/unix/sysv/linux/poll.c:41
#1 0x0000007f8ac27f58 in () at /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0
#2 0x0000005560213bd0 in ()
Spinning. Please run ‘gdb gst-launch-1.0 13487’ to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.
Looking for variation to the pipeline on the forum, I managed to make it work slightly modifying the pipeline, adding a parser and changing the decoder:
gst-launch-1.0 udpsrc port=5000 ! ‘application/x-rtp,encoding-name=H264,payload=96’ ! rtph264depay ! h264parse ! omxh264dec ! autovideosink
but the framerate is nowhere near what I get on the windows 10 workstation. Even though latency is small the framerate looks much lower.
Should I change the pipeline on the Xavier AGX?
Any help is kindly appreciated, thank you all!