Jetson Nano - Limiting the results shown by the DetectNet example.

I have a jetson nano and I’ve been tinkering with it for a long while now. I am currently working with the detectnet sample taken from the samples.

Now, what I want to do is to limit the number of things it detects.

My original idea was to get the frame via OpenCV, and have the OpenCV Frame analyzed by the detectNet Framework, and then cull the result set to just get the classes I want, and then get their corresponding rectangles. I’ll draw the rectangles on my OpenCV frame, and then display that.

However, after running the the net.Detect() code, it seems that it only accepted a data format from the gst Camera.

I’ve also tried to just get the rectangle coordinates I get and then put them on an openCV Mat and display that instead. but to no avail.

How do I proceed with this one?

Also, my PiCam suddenly stopped working? It was working hours ago, and now, I get this error sequence:

[gstreamer] gstreamer v4l2src0 ERROR Internal data stream error.
[gstreamer] gstreamer Debugging info: gstbasesrc.c(3055): gst_base_src_loop (): streaming stopped, reason not-negotiated (-4)
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink

Hi Angelo, are you using one of the SSD-Mobilenet/Inception models? If so, try changing the class names that you want to ignore to ‘void’ (without the ‘’) in the labels file (e.g. /data/networks/SSD-Mobilenet-v2/ssd_coco_labels.txt). The classes with the name void will then be ignored during detection.

You can also use the cudaFromNumpy() routines from jetson.utils to convert numpy ndarray into CUDA memory to pass to the detector. See the cuda-from-numpy.py script for an example. I believe Python cv2 module uses numpy arrays as well. The format you pass to cudaFromNumpy() should be a 4-channel float RGBA ndarray for it to work with the detectNet object.

Have you tried rebooting your system?

Here an example, you need just to edit gst pipeline to read from your usb camera:
https://devtalk.nvidia.com/default/topic/1063275/jetson-nano/-python-how-to-convert-opencv-frame-to-cuda-memory/post/5384478/#5384478

Anyway I suggest you to use yolov3-light

Yeah, I have tried rebooting already. When I call the camera using openCV using:

cap = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)60/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx, width=(int)960, height=(int)616 ! videoconvert ! appsink")

I can open the camera just fine, but with the gstCamera Code, I can’t run it anymore.

I’ll try your suggestion with the numpy array input, and see what happens. Thank you.

Got it to work, I also had to convert the opencv numpy array from BGR to RGBA, this is the code I’m running:

# Get the video input
cap = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)60/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx, width=(int)960, height=(int)616 ! videoconvert ! appsink")

if cap.isOpened():
    while True:
        ret, frame = cap.read()
        frame_rgba = cv2.cvtColor(frame, cv2.COLOR_BGR2RGBA)
        width = frame.shape[1]
        height = frame.shape[0]

        cuda_mem = jetson.utils.cudaFromNumpy(frame)

       detections = net.Detect(cuda_mem, width, height)

I used this post as reference: https://devtalk.nvidia.com/default/topic/1063275/jetson-nano/-python-how-to-convert-opencv-frame-to-cuda-memory/post/5384478/#5384478

I also edited the labels text file and set everything else I didn’t need to void. If you wanna do this, make sure you make a backup of the original labels file.

EDIT: one last question though, how do you disable cudaFromNumpy from logging anything?

Open up your /utils/python/bindings/PyNumPy.cpp file (line 119) and comment this out:

//printf(LOG_PY_UTILS "cudaFromNumpy()  ndarray dim %i = %li\n", n, dims[n]);

Save that file, and then make and sudo make install the project again:

$ cd <jetson-inference>/build
$ make
$ sudo make install

I think that should take care of the output text coming from cudaFromNumpy() function.

Hy angelo_v,
I have also same problem and i tried your code but failed. Can you help me out how can i modify your code because i am using USB Webcam.

Here is the output when i run your code

jetson.inference.init.py
jetson.inference – initializing Python 2.7 bindings…
jetson.inference – registering module types…
jetson.inference – done registering module types
jetson.inference – done Python 2.7 binding initialization
jetson.utils.init.py
jetson.utils – initializing Python 2.7 bindings…
jetson.utils – registering module functions…
jetson.utils – done registering module functions
jetson.utils – registering module types…
jetson.utils – done registering module types
jetson.utils – done Python 2.7 binding initialization
jetson.inference – PyTensorNet_New()
jetson.inference – PyDetectNet_Init()
jetson.inference – detectNet loading network using argv command line params
jetson.inference – detectNet.init() argv[0] = ‘./detectnet-camera.py’

detectNet – loading detection network model from:
– model networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff
– input_blob ‘Input’
– output_blob ‘NMS’
– output_count ‘NMS_1’
– class_labels networks/SSD-Mobilenet-v2/ssd_coco_labels.txt
– threshold 0.500000
– batch_size 1

[TRT] TensorRT version 5.1.6
[TRT] loading NVIDIA plugins…
[TRT] Plugin Creator registration succeeded - GridAnchor_TRT
[TRT] Plugin Creator registration succeeded - NMS_TRT
[TRT] Plugin Creator registration succeeded - Reorg_TRT
[TRT] Plugin Creator registration succeeded - Region_TRT
[TRT] Plugin Creator registration succeeded - Clip_TRT
[TRT] Plugin Creator registration succeeded - LReLU_TRT
[TRT] Plugin Creator registration succeeded - PriorBox_TRT
[TRT] Plugin Creator registration succeeded - Normalize_TRT
[TRT] Plugin Creator registration succeeded - RPROI_TRT
[TRT] Plugin Creator registration succeeded - BatchedNMS_TRT
[TRT] completed loading NVIDIA plugins.
[TRT] detected model format - UFF (extension ‘.uff’)
[TRT] desired precision specified for GPU: FASTEST
[TRT] requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT] native precisions detected for GPU: FP32, FP16
[TRT] selecting fastest native precision for GPU: FP16
[TRT] attempting to open engine cache file networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.1.1.GPU.FP16.engine
[TRT] loading network profile from engine cache… networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.1.1.GPU.FP16.engine
[TRT] device GPU, networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff loaded
[TRT] device GPU, CUDA engine context initialized with 3 bindings
[TRT] binding – index 0
– name ‘Input’
– type FP32
– in/out INPUT
– # dims 3
– dim #0 3 (CHANNEL)
– dim #1 300 (SPATIAL)
– dim #2 300 (SPATIAL)
[TRT] binding – index 1
– name ‘NMS’
– type FP32
– in/out OUTPUT
– # dims 3
– dim #0 1 (CHANNEL)
– dim #1 100 (SPATIAL)
– dim #2 7 (SPATIAL)
[TRT] binding – index 2
– name ‘NMS_1’
– type FP32
– in/out OUTPUT
– # dims 3
– dim #0 1 (CHANNEL)
– dim #1 1 (SPATIAL)
– dim #2 1 (SPATIAL)
[TRT] binding to input 0 Input binding index: 0
[TRT] binding to input 0 Input dims (b=1 c=3 h=300 w=300) size=1080000
[TRT] binding to output 0 NMS binding index: 1
[TRT] binding to output 0 NMS dims (b=1 c=1 h=100 w=7) size=2800
[TRT] binding to output 1 NMS_1 binding index: 2
[TRT] binding to output 1 NMS_1 dims (b=1 c=1 h=1 w=1) size=4
device GPU, networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff initialized.
W = 7 H = 100 C = 1
detectNet – maximum bounding boxes: 100
detectNet – loaded 5 class info entries
detectNet – number of object classes: 5
Gtk-Message: 17:00:49.139: Failed to load module “canberra-gtk-module”
jetson.utils – cudaFromNumpy() ndarray dim 0 = 480
jetson.utils – cudaFromNumpy() ndarray dim 1 = 640
jetson.utils – cudaFromNumpy() ndarray dim 2 = 3
[TRT] cuda/depthwiseConv.cu (581) - Cuda Error in launchKernel<float, float, float>: 4 (unspecified launch failure)
[TRT] cuda/depthwiseConv.cu (581) - Cuda Error in launchKernel<float, float, float>: 4 (unspecified launch failure)
[TRT] detectNet::Detect() – failed to execute TensorRT context
Traceback (most recent call last):
File “./detectnet-camera.py”, line 496, in
detections = net.Detect(img, width, height)
Exception: jetson.inference – detectNet.Detect() encountered an error classifying the image
jetson.utils – freeing CUDA mapped memory
[cuda] cudaFreeHost(ptr)
[cuda] unspecified launch failure (error 4) (hex 0x04)
[cuda] /home/fyp/jetson-inference/utils/python/bindings/PyCUDA.cpp:121
jetson.utils – failed to free CUDA mapped memory with cudaFreeHost()
PyTensorNet_Dealloc()
[cuda] cudaFreeHost(mDetectionSets[0])
[cuda] unspecified launch failure (error 4) (hex 0x04)
[cuda] /home/fyp/jetson-inference/c/detectNet.cpp:66
[cuda] cudaFreeHost(mClassColors[0])
[cuda] unspecified launch failure (error 4) (hex 0x04)
[cuda] /home/fyp/jetson-inference/c/detectNet.cpp:74
[TRT] runtime.cpp (31) - Cuda Error in free: 4 (unspecified launch failure)
terminate called after throwing an instance of ‘nvinfer1::CudaError’
what(): std::exception
Aborted (core dumped)

Thanks in Advance

Hi maaz.bsee55,

Please help to open a new topic for your issue. Thanks