MJPEG to opencv, any other tricks to maximize fps?

I’m working with a USB 2.0 camera that seems to reliably deliver 100 fps to my opencv Python script running on my desktop. I know I’m being greedy, but here goes:

After perusing this forum, I’ve come up with the following pipeline that delivers 85-95 fps on the Jetson Nano in the same Python script:

v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=640, height=480, framerate=61612/513, format=MJPG ! jpegparse ! nvjpegdec ! video/x-raw,format=I420,width=640,height=480,framerate=61612/513 ! videoconvert ! appsink

Is there anything else I can do to squeeze a couple more fps out or am I at the limit?

Thanks!

Hi,
The decoded YUVs are in NVMM buffer and need to copy to CPU buffer and send to appsink, so there is high CPU usage. Please execute sudo nvpmodel -m 0 and sudo jetson_clcoks. See if it makes performance improvement.

1 Like

Thank you, DaneLLL, that did make a difference! It is in the 90+ fps range now, averaging about 95 fps.

Please correct me if I’m wrong: setting nvpmodel to mode 0 removed all constraints on the CPU/GPU, and jetson_clocks then sets the clock speed to maximum as mode 0 has no constraints - the Nano is running with the pedal to the metal?

Begs the question: Is there a more efficient pipeline that doesn’t require warp-factor 10?

It might be a bit more efficient to do YUV → BGRx with nvvidconv and have videoconvert just remove the extra 4th byte:

v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=640, height=480, framerate=61612/513, format=MJPG ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420,width=640,height=480,framerate=61612/513' ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! appsink

Hi, HP:

That yields the following error when run with gst-launch-1.0:

nvbuf_utils: nvbuffer Payload Type not supported

I missed to specify BGR format conversion, it might explain this… Does the following work ?

gst-launch-1.0 -v  v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=640, height=480, framerate=61612/513, format=MJPG ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420,width=640,height=480,framerate=61612/513' ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1

# You may also try nvv4l2decoder instead of nvjpegdec:
gst-launch-1.0 -v  v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=640, height=480, framerate=61612/513, format=MJPG ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1

# You may also try removing jpegparse

If not working, you may have a look to this.

1 Like

Y’all are great!

HP, that last one works a charm! 99+ FPS and ~60% lower CPU load. :)

Thank you very much for the help and the opportunity to learn!

Hi,
For checking effect of sudo nvpmodel -m 0 and sudo jetson_clocks. Please run sudo tegrastats.