Trying to understand jetson.utils.gstCamera

skillet · September 7, 2020, 8:26am

I have previously tinkered with images using OpenCV (like cropping an image, change to gray scale or draw squares and texts), but I am still new to jetson.utils.gstCamera.

Are there any ways to

crop img, from the image obtained from:
camera = jetson.utils.gstCamera(1280,720,‘/dev/video0’)
img, width, height = camera.CaptureRGBA()
I have tried img=img[x1:x2,y1:y2] but doesn’t work. Is it related to CUDA images?
is there any way to convert them to a format that can be handled using OpenCV commands?
draw a square or text similar to cv2 commands
cv2.rectangles(image,point1,point2,color,thickness)
cv2.putText(image,text,point,font,…)

Any help would be greatly appreciated
Thank you in advance

DaneLLL · September 8, 2020, 5:28am

Hi,
We suggest check the source code gstCamera.pp in

github.com

dusty-nv/jetson-utils/blob/cae2730a09edd955631fcbdfc8c4c38556a93600/camera/gstCamera.cpp

/*
 * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.

This file has been truncated. show original

Around line #144, you can see it is to launch a gstreamer pipeline. For customization, you can leverage gstreamer plugins. Please check gstreamer user guide. For doing crpping/scaling/format conversion, you can use nvvidconv plugin.

If your usecase is to run deep learning inference, suggest you check DeepStream SDK.

After the installation, you can see samples in

/opt/nvidia/deepstream/deepstream-5.0/

Please take a look and give it a try.

skillet · September 8, 2020, 6:24am

Dear DaneLLL

Thank you for your help and for the link.
I will have a look at it and probably be back with a ton of questions!

Thank you again!!

skillet · September 8, 2020, 6:56am

Hi again,
Quick one: by any chance is it possible to get the module or a user guide for python?

dusty_nv · September 8, 2020, 3:43pm

@skillet, instead of using gstCamera directly, I would recommend migrating to using the newer videoSource API from jetson-inference - see here for more info:

jetson-inference/aux-streaming.md at master · dusty-nv/jetson-inference · GitHub

There are some Python CUDA-based functions available for cropping, color conversion, resizing, ect that you can see used in the cuda-examples.py sample. For more info, see here:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-image.md

To pass the image buffer to cv2, use the cudaToNumpy() function:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-image.md#converting-to-numpy-arrays

skillet · September 9, 2020, 12:23am

Thanks again, man!
Will try it and let you know how it went.
Cheers!!

skillet · September 10, 2020, 2:19am

Follow up!
It worked great!! Managed to crop the image and perform a simple test!
I really would not have managed that without your help.
Thanks a LOT!