We’re porting an existing system from a Snapdragon board to the Jetson and running into issues with video latency. I’m essentially taking the Argus sample 10 that writes an h264 stream to a file and instead of sending the stream to a file, I’m packaging it up and sending over rtp. I did change the encoder in the sample to use the baseline profile and level 3 (our stream is 640x480), so I’m pretty confident there are no b-frames involved. The encoder is reporting latency of around 35 ms on average, and each NAL gets sent in about 500 microseconds in the dqueue thread, so I’m at a bit of a loss as to where the latency is coming from. The tranfer of video memory to user space, perhaps? The end-to-end latency with my code is 530 ms. Using gstreamer as the source but running the same decoder at the other end, the latency is 330 ms, but we’d like to use Argus, for a variety of reasons, and we’d like to get closer to 200 ms.
A related question: What is the starting time for the timestamp field in the v4l_buf? It would be helpful to know when the frame was captured relative to the system clock.