Try to reduce the conversion time with multiprocessing. But I got this:
If I try to put to a cuda memory frame to a multiprocessing.Queue() (self.q_numpy_frame) in a separate process, then I get “TypeError: can’t pickle PyCapsule objects”
Hi benkelaci, it doesn’t seem like Python multiprocessing package supports PyCapsule objects. Instead, you might want to use Python thread module to spawn threads and pass around the object returned from cudaFromNumpy().
I don’t think what you have there can go any faster in pure Python. Even if you use threading and spawn multiple os threads, thanks to python’s GIL, only one thread will run at a time. Calls to C code like cv2.CvtColor are usually no exception unless they’re writtin a very specific way:
Yeah, I tried it with threading. Similar speed as with multiprocessing (in case where the conversion and inference in the same process)
If you can change the PyCapsule object to any other pickle-compatible one, then we can reach better speed (around 20 FPS) than current one (12 FPS). Possible?
Other idea: I am using a webcamera and I use openCV VideoCapture(), that is why I need to use cudaFromNumpy() function. Is there a good, faster alternative for this approach?
I don’t think that’s possible. Will explain later. On mobile atm.
Edit: I don’t think it’s possible because cudaFromNumpy(color_image) returns an encapsulated pointer inside it’s PyCapsule iirc and even if you could pickle and send that address around, it wouldn’t be very useful to another process.
Then you just built a pipeline with that element at the beginning. No matter what you’re going to have to convert into the format required by the rest of your pipeline (" … ! nvvidconv ! 'video/x-raw(memory:NVMM) ! … ").
You can find some examples from nvidia you can adapt in their accelerated gstreamer user’s guide,
Although I would recommend also doing the Gstreamer tutorials in C, even if you don’t know C, since gstreamer looks like C in any language (so you might as well just use C, or at least know how to do it in C).
Using Nvidia’s DeepStream components for gstreamer, which will be coming to nano soon (hopefully).
It would be something like “rtspsrc ! (so some parsing, decoding, and conversion here) ! nvinfer model-file=birds.caffemodel (more options here) ! … ! somesink”
I found a post here dealing with an rtsp source, and off that thread more examples are linked. The inference won’t work on Nano this second, but you should be able to build your pipeline and have it ready for when DeepStream 4 is released for nano. For example, “gst-launch-1.0 rtspsrc location=rtsp://admin:pass@192.168.30.61/Streaming/Channels/102/ ! rtph264depay ! h264parse ! omxh264dec ! nvoverlaysink” should work on the nano right now to decode and playback an rtsp stream (example from tha thread).