videoSource.Capture()'s cudaImage to numpy

drakorg · July 9, 2020, 6:03am

Hi, I’m trying to make use of the videoSource component available at the dev branch of jetson-utils, but I can’t find a way to convert its result (cudaImage) to a numpy array … I’m able to inspect this cudaImage properties (width, height, channels, ptr, format, mapped) and it appears to be a valid frame and everything, but apparently I can’t feed it to cudaToNumpy() because its not mapped (it says mapped=false) … while other examples I’ve checked are able to convert cudaImages to numpy successfully. The only difference that I found is that the cudaImages that were converted succesfully have the mapped flag in true (as they were created for example via cudaAllocMapped.

Is there a way to turn an unmapped cudaImage to a mapped one so that I can then convert it to numpy? Would that be the way to go?

Here’s an example:

>>> cudaImage_mapped = jetson.utils.cudaAllocMapped(width=1280, height=960, format='rgb32f')
>>> str(cudaImage_mapped)
'<cudaImage object>\n   -- ptr:      0x101370000\n   -- size:     14745600\n   -- width:    1280\n   -- height:   960\n   -- channels: 3\n   -- format:   rgb32f\n   -- mapped:   true\n   -- freeOnDelete: true\n'
>>> numpy_mapped = jetson.utils.cudaToNumpy(cudaImage_mapped)
>>> cudaImage_not_mapped = vs.Capture() # vs is a jetson.utils.videoSource instance
>>> str(cudaImage_not_mapped)
'<cudaImage object>\n   -- ptr:      0xf00db0000\n   -- size:     3686400\n   -- width:    1280\n   -- height:   960\n   -- channels: 3\n   -- format:   rgb8\n   -- mapped:   false\n   -- freeOnDelete: false\n'
>>> numpy_not_mapped = jetson.utils.cudaToNumpy(cudaImage_not_mapped)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: jetson.utils -- cudaToNumpy() needs to use CUDA mapped memory as input (allocate with mapped=1)

Thank you.
Regards.

AastaLLL · July 9, 2020, 8:31am

Hi,

github.com

dusty-nv/jetson-utils/blob/master/video/videoSource.h#L203


	 * @returns `true` if a frame was captured, `false` if there was an error or a timeout occurred.
	 */
	template<typename T> bool Capture( T** image, uint64_t timeout=UINT64_MAX )		{ return Capture((void**)image, imageFormatFromType<T>(), timeout); }
	
	/**
	 * Capture the next image from the video stream.
	 *
	 * The image formats supported by Capture() are `IMAGE_RGB8` (uchar3), `IMAGE_RGBA8` (uchar4), 
	 * `IMAGE_RGB32F` (float3), and `IMAGE_RGBA32F` (float4). @see imageFormat for more info.
	 *
	 * @param[out] image output pointer that will be set to the memory containing the image.
 	 *                   If this interface has it's videoOptions::zeroCopy flag set to true,
	 *                   the memory was allocated in mapped CPU/GPU memory and is be accessible
	 *                   from both CPU and CUDA.  Otherwise, it's accessible only from CUDA.
	 *
	 * @param[in] timeout timeout in milliseconds to wait to capture the image before returning.
	 *                    A timeout value of `UINT64_MAX` (the default) will wait forever, and
	 *                    a timeout of 0 will return instantly if a frame wasn't immediately ready.
	 *
	 * @returns `true` if a frame was captured, `false` if there was an error or a timeout occurred.
	 */

Suppose you can get a mapped memory if the configure is well set.
Would you mind to check it further.

Thanks.

dusty_nv · July 9, 2020, 4:08pm

Hmm, what URI are you creating your videoSource with? I think by default I create them all to use mapped memory…so curious why it is not already mapped.

drakorg · July 9, 2020, 8:47pm

Hi, it’s an rtsp source. The videoSource was created like this:

>>> vs = jetson.utils.videoSource('rtsp://user:pass@192.168.0.20:554', sys.argv)

The whole snippet would be:

>>> cudaImage_mapped = jetson.utils.cudaAllocMapped(width=1280, height=960, format='rgb32f')
>>> str(cudaImage_mapped)
'<cudaImage object>\n   -- ptr:      0x101370000\n   -- size:     14745600\n   -- width:    1280\n   -- height:   960\n   -- channels: 3\n   -- format:   rgb32f\n   -- mapped:   true\n   -- freeOnDelete: true\n'
>>> numpy_mapped = jetson.utils.cudaToNumpy(cudaImage_mapped)

>>> vs = jetson.utils.videoSource('rtsp://user:pass@192.168.0.20:554', sys.argv)
>>> vs.Open()
>>> cudaImage_not_mapped = vs.Capture() # vs is a jetson.utils.videoSource instance
>>> str(cudaImage_not_mapped)
'<cudaImage object>\n   -- ptr:      0xf00db0000\n   -- size:     3686400\n   -- width:    1280\n   -- height:   960\n   -- channels: 3\n   -- format:   rgb8\n   -- mapped:   false\n   -- freeOnDelete: false\n'
>>> numpy_not_mapped = jetson.utils.cudaToNumpy(cudaImage_not_mapped)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: jetson.utils -- cudaToNumpy() needs to use CUDA mapped memory as input (allocate with mapped=1)

dusty_nv · July 16, 2020, 5:54am

OK gotcha, yep it was a bug - thanks for letting me know. I just fixed it in the dev branch - if you run this and pull the latest, it should work now:

$ cd jetson-inference
$ git pull origin dev
$ cd build
$ cmake ../
$ make
$ sudo make install

drakorg · July 9, 2020, 10:30pm

Great news! it works now, thank you!
Best regards.