No difference between CPU and Jetson nano

Hi… I am running face detection python code in CPU. I am obtaining FPS of ~45. while running the same code on Jetson nano, there is no improvement of FPS. what to do to overcome this issue. I have tried some other python programs in jetson nano. but there is no difference. someone Please help me…

Your question is not clear. What code you are running and with what CPU? Without this information FPS does not mean anything.

If you are comparing performance of Jetson Nano CPU and GPU with right batch size and resolution, there is should be huge difference, otherwise you running on CPU in both cases.

If you are comparing Jetson Nano GPU to powerful desktop CPU, then you are not correctly comparing them. Jetson Nano consumes 10W but desktop CPU usually consumes order of magnitude more power. For example, if you have the same performance with 10W GPU in Jetson Nano and 100W desktop CPU that means Jetson Nano is 10 times better. Not to mention the price difference between desktop PC and Jetson Nano, especially 2GB version.

Thanks for your response. I am running opencv dnn based face detection code in windows 10 pro, intel core i5-9400f CPU@2.9 ghz processor with 16 GB RAM. With this I am getting 45 FPS. While running the same in jetson nano B01 module, I can get ~45 fps. Another example, with yolov4 person detection i am getting ~4FPS in both jetson nano and windows CPU.

Jetson nano has jetpack version of 4.4.1 and opencv 4.1.1 by default

  1. If I want to improve FPS in yolov4 based person detection what should i do?
  2. How can I confirm jetson is using CPU or GPU?
  3. Is there any changes i have to make in code or settings to work with jetson GPU?

I suggest to install jtop, it will show a lot of useful information including CPU/GPU load: https://pypi.org/project/jetson-stats/. If GPU is fully loaded when running your code and you are using maximum power (NV Power in jtop should be “MAXN”), you will know you are using GPU efficiently.

I did not try using OpenCV myself yet but in this thread mdegans wrote that “OpenCV installed by default on nano does not have CUDA built in. You can see with print(cv2.getBuildInformaton()) in Python or (or cv2::getBuildInformation() in C/c++ probably. Almost all the fun build options are off.”. In other words, you have to build OpenCV with GPU support to get GPU acceleration in OpenCV.

UPDATE:
I looked around for more information about OpenCV on Jetson Nano since in the near future I planned to use it too, so I decided it is good time to figure out how to get it working with GPU. To check if OpenCV has cuda, run python3 and execute the following code:

import cv
print(cv2.cuda.getCudaEnabledDeviceCount())

If output is zero, then OpenCV needs to be built with GPU support. Here is an article how to install OpenCV 4.5 on Jetson Nano, it contains a lot of details how to install all the requirements to build OpenCV and how to compile it with GPU support.

Ok…Thanks for your timely support. I will try those things and get you back

While running the code CPU is fully loaded but not the GPU. NV Power[0] is MAXN in jtop and also opencv is compiled with CUDA. I am getting 1 for print(cv2.cuda.getCudaEnabledDeviceCount())… FPS is still very much low. what would be the reason?

Hmm… other users here may be more familiar with openCV so my comments may not be that correct.

  1. Please check if your code implementation is really using CUDA. You compiled openCV source with CUDA does not represent your program will use CUDA to run…
    For example, openCV has “Mat” and “GpuMat” to handle data. Only those data stored by GpuMat is able to passed and run by cuda kernel. If it is just handled by “Mat”, no matter how many configuration you are done in system side (MaxN), it will not run by GPU.

  2. You can use tegrastats tool to see the gpu loading. The value after GR3D_FREQ is the loading of GPU.

nvidia@tegra-ubuntu:~$ sudo tegrastats
[sudo] password for nvidia:
RAM 601/1972MB (lfb 206x4MB) SWAP 0/986MB (cached 0MB) IRAM 0/252kB(lfb 252kB) CPU [6%@102,8%@102,1%@102,3%@102] EMC_FREQ 2%@1600 GR3D_FREQ 0%@76 APE 25 PLL@30C CPU@32C PMIC@100C GPU@32C AO@37.5C thermal@32C

Anyway thanks for your suggestion… I checked using tegrastats too. Is there any changes i have to make in program to run in GPU?. Any help will be appreciable

If you want other users to help review and give suggestion of your program, then what you need to do is share your current code to let other check. Maybe you can dig into your code and tell us which line you think should be running on GPU but it does not happen.

or maybe you can ask this question in opencv forum. If you just want their sample code to run with CUDA enabled, they may give you a quick answer.