How to integrate csi-camera with yolov5 model to run on Jetson Nano

rusaliapple2801 · March 20, 2023, 8:22am

Hello,
I have been trying to run csi-camera to detect custom objects in real time using yolov5. I have read in a lot of previous related issues that it we can use this code from repo

But I am unable to figure out how to integrate this code with the yolov5 detect.py file.

Can someone please help me understand how to do this?

Here are my files:
detect.py (14.0 KB)

dataloaders.py (54.4 KB)

AastaLLL · March 20, 2023, 9:00am

Hi,

You can find jetson inference or Deepstream to run TensorRT with a CSI camera.

Jetson inference: GitHub - dusty-nv/jetson-inference: Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
Deepstream: https://developer.nvidia.com/deepstream-sdk

Thanks.

rusaliapple2801 · March 20, 2023, 9:30am

Hello, thank you for helping.
I went through the jetson inference repo, but I can’t seem to find the option for yolov5 model.
I am also encountering some errors while trying to accomplish my objective using deepstream.

Can you please suggest if I can use the first method I mentioned in my issue, to use the csi-camera code for the yolo model? If it is possible, can you please help me understand how can I do it?

AastaLLL · March 21, 2023, 5:20am

Hi,

Have you wrapped the CSI frame into a cvMat format?
If yes, you can follow the below sample to copy the buffer to GPU and then infer it with TensorRT.

https://elinux.org/Jetson/L4T/TRT_Customized_Example#OpenCV_with_PLAN_model

Thanks.

rusaliapple2801 · March 21, 2023, 7:09am

No, I did not wrap the CSI frame into a cvMat format. I am unable to understand what that is and why or how should I do it. Can you kindly help me understand better?
Thank you

AastaLLL · March 22, 2023, 3:14am

Hi,

github.com

JetsonHacksNano/CSI-Camera/blob/master/simple_camera.py#L55


      
          def show_camera():
              window_title = "CSI Camera"
          
          
    # To flip the image, modify the flip_method parameter (0 and 2 are the most common)
              print(gstreamer_pipeline(flip_method=0))
              video_capture = cv2.VideoCapture(gstreamer_pipeline(flip_method=0), cv2.CAP_GSTREAMER)
              if video_capture.isOpened():
                  try:
                      window_handle = cv2.namedWindow(window_title, cv2.WINDOW_AUTOSIZE)
                      while True:
                          ret_val, frame = video_capture.read()
                          # Check to see if the user closed the window
                          # Under GTK+ (Jetson Default), WND_PROP_VISIBLE does not work correctly. Under Qt it does
                          # GTK - Substitute WND_PROP_AUTOSIZE to detect if window has been closed by user
                          if cv2.getWindowProperty(window_title, cv2.WND_PROP_AUTOSIZE) >= 0:
                              cv2.imshow(window_title, frame)
                          else:
                              break 
                          keyCode = cv2.waitKey(10) & 0xFF
                          # Stop the program on the ESC key or 'q'
                          if keyCode == 27 or keyCode == ord('q'):

The frame variable here is the cvMat format already.
You can pass it to TensorRT like:

np.copyto(host_inputs[0], frame.ravel())

Thanks.

rusaliapple2801 · March 22, 2023, 9:19am

Hello @AastaLLL ,
Okay I understood from your suggested link where I can pass this frame. But in the link, the /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx --saveEngine=sample.engine the instruction is for resnet-50. Can you please tell me how can I modify it for yolov5 weights?

AastaLLL · March 23, 2023, 5:01am

Hi,

TensorRT needs ONNX format so you will need to convert it to ONNX file first.

Is your model defined with .cfg and .weight, the darknet format?
If yes, there are some samples online to convert the model format, for example:

Thanks.

rusaliapple2801 · March 23, 2023, 7:11am

Hello @AastaLLL ,
I have converted my weights to ONNX format. But I am unable to figure how to run yolov5 model instead of Resnet50. As mentioned your suggested link, I tried to navigate to the /usr/src/tensorrt/data directory, to change the Resnet50 to Yolov5, but I couldn’t find yolov5 in the options. Please tell me how to run yolov5 like Resnet50 using this code /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/resnet50/ResNet50.onnx --saveEngine=sample.engine.

UPDATE: I ran the code for building the engine using my yolov5 onnx weights. I have built a sample engine. Now when I try to run the code mentioned in your link, on a python terminal, I get this error for tesorrt.
Although I have duely followed all steps and installed tensorrt on my jetson nano.

I can’t seem to understand why tensorrt is not being recognised in the python environment.

Python - 3.9

Please help me out!

rusaliapple2801 · March 24, 2023, 5:51am

@AastaLLL Can you please look into the issue and help me out? Its really urgent for me to solve this!!

AastaLLL · March 27, 2023, 6:23am

Hi,

We only support the default python version. For JetPack4, it’s python 3.6.
For other python versions, you can build it from the source:

You can infer the model with the sample shared above.
For YOLOv5, you will need some post-processing to convert the output tensor into bounding boxes.
You can check the author’s code or some community implementation for more info.

Thanks.

system · April 26, 2023, 2:36am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.