videoSource.Capture()'s cudaImage to numpy

drakorg · July 9, 2020, 6:03am

Hi, I’m trying to make use of the videoSource component available at the dev branch of jetson-utils, but I can’t find a way to convert its result (cudaImage) to a numpy array … I’m able to inspect this cudaImage properties (width, height, channels, ptr, format, mapped) and it appears to be a valid frame and everything, but apparently I can’t feed it to cudaToNumpy() because its not mapped (it says mapped=false) … while other examples I’ve checked are able to convert cudaImages to numpy successfully. The only difference that I found is that the cudaImages that were converted succesfully have the mapped flag in true (as they were created for example via cudaAllocMapped.

Is there a way to turn an unmapped cudaImage to a mapped one so that I can then convert it to numpy? Would that be the way to go?

Here’s an example:

>>> cudaImage_mapped = jetson.utils.cudaAllocMapped(width=1280, height=960, format='rgb32f')
>>> str(cudaImage_mapped)
'<cudaImage object>\n   -- ptr:      0x101370000\n   -- size:     14745600\n   -- width:    1280\n   -- height:   960\n   -- channels: 3\n   -- format:   rgb32f\n   -- mapped:   true\n   -- freeOnDelete: true\n'
>>> numpy_mapped = jetson.utils.cudaToNumpy(cudaImage_mapped)
>>> cudaImage_not_mapped = vs.Capture() # vs is a jetson.utils.videoSource instance
>>> str(cudaImage_not_mapped)
'<cudaImage object>\n   -- ptr:      0xf00db0000\n   -- size:     3686400\n   -- width:    1280\n   -- height:   960\n   -- channels: 3\n   -- format:   rgb8\n   -- mapped:   false\n   -- freeOnDelete: false\n'
>>> numpy_not_mapped = jetson.utils.cudaToNumpy(cudaImage_not_mapped)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: jetson.utils -- cudaToNumpy() needs to use CUDA mapped memory as input (allocate with mapped=1)

Thank you.
Regards.

AastaLLL · July 9, 2020, 8:31am

Hi,

github.com

dusty-nv/jetson-utils/blob/master/video/videoSource.h#L203


	 * @returns `true` if a frame was captured, `false` if there was an error or a timeout occurred.
	 */
	template<typename T> bool Capture( T** image, uint64_t timeout=UINT64_MAX )		{ return Capture((void**)image, imageFormatFromType<T>(), timeout); }
	
	/**
	 * Capture the next image from the video stream.
	 *
	 * The image formats supported by Capture() are `IMAGE_RGB8` (uchar3), `IMAGE_RGBA8` (uchar4), 
	 * `IMAGE_RGB32F` (float3), and `IMAGE_RGBA32F` (float4). @see imageFormat for more info.
	 *
	 * @param[out] image output pointer that will be set to the memory containing the image.
 	 *                   If this interface has it's videoOptions::zeroCopy flag set to true,
	 *                   the memory was allocated in mapped CPU/GPU memory and is be accessible
	 *                   from both CPU and CUDA.  Otherwise, it's accessible only from CUDA.
	 *
	 * @param[in] timeout timeout in milliseconds to wait to capture the image before returning.
	 *                    A timeout value of `UINT64_MAX` (the default) will wait forever, and
	 *                    a timeout of 0 will return instantly if a frame wasn't immediately ready.
	 *
	 * @returns `true` if a frame was captured, `false` if there was an error or a timeout occurred.
	 */

Suppose you can get a mapped memory if the configure is well set.
Would you mind to check it further.

Thanks.

dusty_nv · July 9, 2020, 4:08pm

Hmm, what URI are you creating your videoSource with? I think by default I create them all to use mapped memory…so curious why it is not already mapped.

drakorg · July 9, 2020, 8:47pm

Hi, it’s an rtsp source. The videoSource was created like this:

>>> vs = jetson.utils.videoSource('rtsp://user:pass@192.168.0.20:554', sys.argv)

The whole snippet would be:

>>> cudaImage_mapped = jetson.utils.cudaAllocMapped(width=1280, height=960, format='rgb32f')
>>> str(cudaImage_mapped)
'<cudaImage object>\n   -- ptr:      0x101370000\n   -- size:     14745600\n   -- width:    1280\n   -- height:   960\n   -- channels: 3\n   -- format:   rgb32f\n   -- mapped:   true\n   -- freeOnDelete: true\n'
>>> numpy_mapped = jetson.utils.cudaToNumpy(cudaImage_mapped)

>>> vs = jetson.utils.videoSource('rtsp://user:pass@192.168.0.20:554', sys.argv)
>>> vs.Open()
>>> cudaImage_not_mapped = vs.Capture() # vs is a jetson.utils.videoSource instance
>>> str(cudaImage_not_mapped)
'<cudaImage object>\n   -- ptr:      0xf00db0000\n   -- size:     3686400\n   -- width:    1280\n   -- height:   960\n   -- channels: 3\n   -- format:   rgb8\n   -- mapped:   false\n   -- freeOnDelete: false\n'
>>> numpy_not_mapped = jetson.utils.cudaToNumpy(cudaImage_not_mapped)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: jetson.utils -- cudaToNumpy() needs to use CUDA mapped memory as input (allocate with mapped=1)

dusty_nv · July 16, 2020, 5:54am

OK gotcha, yep it was a bug - thanks for letting me know. I just fixed it in the dev branch - if you run this and pull the latest, it should work now:

$ cd jetson-inference
$ git pull origin dev
$ cd build
$ cmake ../
$ make
$ sudo make install

drakorg · July 9, 2020, 10:30pm

Great news! it works now, thank you!
Best regards.

avidan2012 · October 11, 2020, 9:41am

hi, this failed for me (jatson nano)

[ 34%] Building NVCC (Device) object CMakeFiles/jetson-inference.dir/c/jetson-inference_generated_tensorConvert.cu.o
In file included from /home/jetbot/workspace/jetson-inference/c/tensorConvert.cu:23:0:
/home/jetbot/workspace/jetson-inference/c/tensorConvert.h:28:10: fatal error: imageFormat.h: No such file or directory
 #include "imageFormat.h"
          ^~~~~~~~~~~~~~~
compilation terminated.
CMake Error at jetson-inference_generated_tensorConvert.cu.o.cmake:219 (message):
Error generating
/home/jetbot/workspace/jetson-inference/build/CMakeFiles/jetson-inference.dir/c/./jetson-inference_generated_tensorConvert.cu.o


CMakeFiles/jetson-inference.dir/build.make:501: recipe for target 'CMakeFiles/jetson-inference.dir/c/jetson-inference_generated_tensorConvert.cu.o' failed
make[2]: *** [CMakeFiles/jetson-inference.dir/c/jetson-inference_generated_tensorConvert.cu.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/jetson-inference.dir/all' failed
make[1]: *** [CMakeFiles/jetson-inference.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

dusty_nv · October 12, 2020, 5:23pm

Hi @avidan2012, I think that your jetson-inference/utils submodule hadn’t been updated. Can you try updating your repo?

$ cd jetson-inference/
$ git submodule foreach git pull origin master
$ cd build/
$ cmake ../
$ make
$ sudo make install

If that still doesn’t work, then you may want to totally re-clone a fresh copy of the repo from master.

avidan2012 · October 12, 2020, 5:39pm

thanks dusty for the super quick response :)
i will update in case i face other issues.