Hardware Platform: Jetson Xavier AGX & Xavier NX
DeepStream Version: 6.0.1
JetPack Version: 4.6.3
TensorRT Version: 8.2.1.9-1+cuda10.2
Issue Type" question
This is a follow up question to: NVEGLGLESSINK is slow on xavier nx
The network that I’m running (a custom version of yolov5) doesn’t run real-time (I don’t need it to), in a pipeline that is based on the pipeline in the deepstream-test3 app. When I play a video file (filesrc), the NX consistently is able to process ~7-9 fps and AGX processes ~18-19 fps which is sufficient for what I need (both measured when interval = 0). When I switch from a filesrc to a udpsrc, processing starts to be very inconsistent, varying from 7-9 fps down to 2-3 fps and back on the NX and varying from 18-19 fps on the AGX down to 4-5fps, even when setting interval to something that should allow inference to keep up.
My ultimate goal is to get the pipeline running so that inference is being performed as fast as available processing will allow (not necessarily on every frame, just whenever one frame is done, start inference on the most recent frame received. I would like the rate at which frames are processed to be relatively consistent and ideally the display to be smooth, displaying every frame, regardless of whether or not inference was performed on it, without dropping buffers.
I’ve discovered if I set interval to a really high number (like 100,000) so as to effectively disable inference, I get the behavior that for a filesrc, the display sink works as it should, but for a udpsrc, the display sink drops frames and the display is choppy, so I think there’s something in my pipeline in general that isn’t set up correctly for a udp source. Both the video source and udp source are 1280 x 720, 25 fps mpeg ts.
Edit: I just tested the deepstream-test3 app with it’s default configuration and I get the exact same behavior: a filesrc plays properly and the display sink doesn’t drop any frames (I think it’s using the default resnet10 which appears to be able to keep up with real time), however on a udpsrc I get the same warning that I get in my pipeline, this time from element nvvideo-renderer: “A lot of buffers are being dropped” and the video display is very choppy as above.
Edit: if I use the command line gst with playbin, i.e. gst-launch-1.0 playbin uri=udp://... then the udp source works as expected, although obviously no inference takes place. No frames dropped on the display sink.
I’ve uploaded the pipelines for all three cases - my pipeline from filesrc, udpsrc and from playbin. I think I have two issues to solve 1) get udpsrc working in the pipeline correctly so even when inference is “turned off” by setting the interval to a really high number, frames on the display sink aren’t dropped, and 2) how to get the pipeline to cleanly handle inference time that is slower that frame rate.
