CUDA error with JP-5.0.2 Update

Hello,

I used to work with JP-5.0.1 version for my project and everything had been going well. After I flashed my Orin with JP-5.0.2, I started having same errors on same code after even many times of reflashing.
I am not sure the problem is about CUDA, TensorRT or Jetson-Inference.

-Project includes multi-threading image processing and shows the results with double screen
-Switched to TensorRT-8.4 from 8.3 with JP update
-Installed Deep-learning packages with sudo apt install nvidia-jetpack (Not with SDK Manger because it always gives ip error.)
-Installed OpenCV with different installer.


Screenshot from 2022-09-15 10-46-40

install_opencv4.6.0_Jetson.sh (3.5 KB)

Hi,

Have you recompiled jetson-inference after upgrading to JetPack 5.0.2?
If yes, could you share the steps in detail to reproduce the error?

Thanks.

Hi,

I actually didin’t upgrade to JetPack 5.0.2, I flashed AGX Orin with SDK Manager by using JetPack 5.0.2 So I installed jetson-inference and compiled from begining.

My steps

  1. Flashed AGX Orin with JetPack 5.0.2 (Only Jetson OS part is choosen, has not choosen Jetson Components part which includes Cuda Cuda-X etc…)

  2. In terminal, I typed sudo apt update & sudo apt install nvidia-jetpack (With this, cuda and tensorrt was installed)

  3. I added Cuda to path

$ export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}
$ export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64
{LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}})

4)I installed OpenCV with " install_opencv4.6.0_Jetson.sh" file which is included at my first post. It includes all the flags, paths etc…

5)I installed Jetson_Inference with the way " Building the Project from Source "

None of these steps gave any error. 100% succes. But when I try to run my code, I got these errors. Most probably about TensorRT

thanks.

Hi @furkankoc22, are you able to run detectnet/detectnet.py from jetson-inference on some test images or camera stream? Or does this error only happen with your application?

1 Like

Hi sir, thanks for replying.
Yes, I am able to run it in tests. Error happens with my project.

  1. First problem is {int(net.GetNetworkFPS())} returns infinitive value and gives error related with cuda like this.
  • With try except it works. It keeps giving same cuda error but works.
  1. Second problem is about net = jetson.inference.detectNet(‘ssd-mobilenet-v2’, threshold=0.5)

Here I add simplified project. Same structure and it has all same errors with my project.
simplified.py (2.8 KB)

Hi @furkankoc22, do you get the errors if you only one of the camera threads? I am curious if the errors have something to do with the multithreading.

Hi @dusty_nv

It works so fine with;
only one camera thread, and removing this line
##cv2.putText(img, f’FPS: {int(net.GetNetworkFPS())}', (30, 30), cv2.FONT_HERSHEY_DUPLEX, 1, (255, 0, 0), 2)

I think the reason that it does work when you create detectNet object as a global, is because you are using python threads (i.e. multiple processes) and CUDA isn’t cross-process like that.

In your webcam class, can you try moving the FPS stuff outside of the for d in detections loop?

        for d in detections:
            try:
                cv2.putText(img, f'FPS: {int(net.GetNetworkFPS())}', (30, 30),cv2.FONT_HERSHEY_DUPLEX, 1, (255, 0, 0), 2)
            except:
                pass

i.e. instead of the above, like this:

        try:
                cv2.putText(img, f'FPS: {int(net.GetNetworkFPS())}', (30, 30),cv2.FONT_HERSHEY_DUPLEX, 1, (255, 0, 0), 2)
        except:
                pass

        for d in detections:
                ...
1 Like

Thank you @dusty_nv