I have been working on an app which uses OpenCV and a USB webcam to run some GPU-based processing on live images. When I use OpenCV’s capture classes to get a frame from the webcam, it takes 30-50 ms to get a frame. This seems insanely long, so I’m wondering what fast configurations of hardware and software are recommended. I think that part of the problem is that the camera is interfacing over USB (so a faster camera solution would be helpful), but the bigger issue as far as I can tell is software: a simple OpenCV app that just reads camera frames (without displaying it or doing anything else) is much slower than a default camera viewer app, even at much smaller resolutions.
So, what cameras are known to be fast and work on the Jetson? And how can I capture video frames and store them in OpenCV structures without having such a huge delay?
I did not attempt to manually change the CPU settings, but when I have an app running that captures images and then does some GPU processing in a loop it only uses around 50% of one core (as reported by System Monitor). This makes me think that CPU capability is not the issue.
Typically, the timing of the frames is directly dependent on the rate at which the camera (webcams are typically slow). For example, I have built an application that supports two 1080p 60 fps cameras using SDI. These cameras have wide support for frame rates and raster. Simple case, when set for 1080p 30 frames arrive about every 33 mS and at 60 fps they arrive about every 16mS. Switch to a basic HD webcam, and at 1080p frame rates drop from 30 to as little as 10 fps. So your receiving frames at 30mS to 50mS tells me your frame rate is between 20 and 30 fps.
In my experience, web cams are notoriously inconsistent in thei behavior. Their ability to produce frames can vary wildly from what they claim. So you are right it is not a CPU issue or even a GPU issue it is a fps issue