Xavier Nx runs yolov8's C++demo and returns an empty result

Run the ONNX model exported from YOLOv8 using Opencv4.9.0, use CUDA, and the result is empty. By using the CPU, the result can be returned.

root@nvidia-desktop:~# nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Sun_Oct_23_22:16:07_PDT_2022
Cuda compilation tools, release 11.4, V11.4.315
Build cuda_11.4.r11.4/compiler.31964100_0
root@nvidia-desktop:~/esdk_deploy/yolov8/ultralytics/examples/YOLOv8-CPP-Inference/build# ./Yolov8CPPInference 
opencv version = 4.9.0

Running on CUDA
[DEBUG:0@1.208] global parallel.cpp:107 createDefaultParallelForAPI core(parallel): Initializing parallel backend...
[DEBUG:0@1.208] global registry_parallel.impl.hpp:69 ParallelBackendRegistry core(parallel): Builtin backends(3): ONETBB(1000); TBB(990); OPENMP(980)
[DEBUG:0@1.208] global registry_parallel.impl.hpp:94 ParallelBackendRegistry core(parallel): Available backends(3): ONETBB(1000); TBB(990); OPENMP(980)
[ INFO:0@1.208] global registry_parallel.impl.hpp:96 ParallelBackendRegistry core(parallel): Enabled backends(3, sorted by priority): ONETBB(1000); TBB(990); OPENMP(980)
[DEBUG:0@1.208] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: ONETBB (priority=1000)
[DEBUG:0@1.208] global plugin_parallel_wrapper.impl.hpp:219 getPluginCandidates core(parallel): ONETBB plugin's glob is 'libopencv_core_parallel_onetbb*.so', 1 location(s)
[DEBUG:0@1.214] global plugin_parallel_wrapper.impl.hpp:229 getPluginCandidates     - /usr/local/opencv44/lib: 0
[DEBUG:0@1.214] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 0 plugin(s) for ONETBB
[DEBUG:0@1.214] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: TBB (priority=990)
[DEBUG:0@1.214] global plugin_parallel_wrapper.impl.hpp:219 getPluginCandidates core(parallel): TBB plugin's glob is 'libopencv_core_parallel_tbb*.so', 1 location(s)
[DEBUG:0@1.217] global plugin_parallel_wrapper.impl.hpp:229 getPluginCandidates     - /usr/local/opencv44/lib: 0
[DEBUG:0@1.217] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 0 plugin(s) for TBB
[DEBUG:0@1.217] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: OPENMP (priority=980)
[DEBUG:0@1.217] global plugin_parallel_wrapper.impl.hpp:219 getPluginCandidates core(parallel): OPENMP plugin's glob is 'libopencv_core_parallel_openmp*.so', 1 location(s)
[DEBUG:0@1.220] global plugin_parallel_wrapper.impl.hpp:229 getPluginCandidates     - /usr/local/opencv44/lib: 0
[DEBUG:0@1.220] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 0 plugin(s) for OPENMP
[DEBUG:0@1.220] global parallel.cpp:93 createParallelForAPI core(parallel): fallback on builtin code
[ INFO:0@4.471] global op_cuda.cpp:80 initCUDABackend CUDA backend will fallback to the CPU implementation for the layer "_input" of type __NetInputLayer__

yolov8 = 1
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
Number of detections:1



YOLOv8-CPP-Inference.zip (175.8 KB)

Here is the source code for C++

Hi,

Could you try to run the sample on a desktop environment?
This can help to figure out the issue comes from Jetson or sample itself.

Thanks.

Thank you for your reply! The previous one was the result of the run. I will run it again as follows:

root@nvidia-desktop:~/esdk_deploy/yolov8/ultralytics/examples/YOLOv8-CPP-Inference/build# ./Yolov8CPPInference 
opencv version = 4.9.0

Running on CUDA
[DEBUG:0@1.213] global parallel.cpp:107 createDefaultParallelForAPI core(parallel): Initializing parallel backend...
[DEBUG:0@1.213] global registry_parallel.impl.hpp:69 ParallelBackendRegistry core(parallel): Builtin backends(3): ONETBB(1000); TBB(990); OPENMP(980)
[DEBUG:0@1.213] global registry_parallel.impl.hpp:94 ParallelBackendRegistry core(parallel): Available backends(3): ONETBB(1000); TBB(990); OPENMP(980)
[ INFO:0@1.213] global registry_parallel.impl.hpp:96 ParallelBackendRegistry core(parallel): Enabled backends(3, sorted by priority): ONETBB(1000); TBB(990); OPENMP(980)
[DEBUG:0@1.213] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: ONETBB (priority=1000)
[DEBUG:0@1.213] global plugin_parallel_wrapper.impl.hpp:219 getPluginCandidates core(parallel): ONETBB plugin's glob is 'libopencv_core_parallel_onetbb*.so', 1 location(s)
[DEBUG:0@1.217] global plugin_parallel_wrapper.impl.hpp:229 getPluginCandidates     - /usr/local/opencv44/lib: 0
[DEBUG:0@1.217] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 0 plugin(s) for ONETBB
[DEBUG:0@1.217] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: TBB (priority=990)
[DEBUG:0@1.217] global plugin_parallel_wrapper.impl.hpp:219 getPluginCandidates core(parallel): TBB plugin's glob is 'libopencv_core_parallel_tbb*.so', 1 location(s)
[DEBUG:0@1.220] global plugin_parallel_wrapper.impl.hpp:229 getPluginCandidates     - /usr/local/opencv44/lib: 0
[DEBUG:0@1.220] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 0 plugin(s) for TBB
[DEBUG:0@1.220] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: OPENMP (priority=980)
[DEBUG:0@1.220] global plugin_parallel_wrapper.impl.hpp:219 getPluginCandidates core(parallel): OPENMP plugin's glob is 'libopencv_core_parallel_openmp*.so', 1 location(s)
[DEBUG:0@1.223] global plugin_parallel_wrapper.impl.hpp:229 getPluginCandidates     - /usr/local/opencv44/lib: 0
[DEBUG:0@1.223] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 0 plugin(s) for OPENMP
[DEBUG:0@1.224] global parallel.cpp:93 createParallelForAPI core(parallel): fallback on builtin code
[ INFO:0@4.254] global op_cuda.cpp:80 initCUDABackend CUDA backend will fallback to the CPU implementation for the layer "_input" of type __NetInputLayer__

yolov8 = 1
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
x= 0
y= 0
w= 0
h= 0
Number of detections:1

I cherish your reply very much. I will post the result log of running using CPU as follows:

root@nvidia-desktop:~/esdk_deploy/yolov8/ultralytics/examples/YOLOv8-CPP-Inference/build# ./Yolov8CPPInference 
opencv version = 4.9.0

Running on CPU
[DEBUG:0@1.172] global parallel.cpp:107 createDefaultParallelForAPI core(parallel): Initializing parallel backend...
[DEBUG:0@1.172] global registry_parallel.impl.hpp:69 ParallelBackendRegistry core(parallel): Builtin backends(3): ONETBB(1000); TBB(990); OPENMP(980)
[DEBUG:0@1.172] global registry_parallel.impl.hpp:94 ParallelBackendRegistry core(parallel): Available backends(3): ONETBB(1000); TBB(990); OPENMP(980)
[ INFO:0@1.172] global registry_parallel.impl.hpp:96 ParallelBackendRegistry core(parallel): Enabled backends(3, sorted by priority): ONETBB(1000); TBB(990); OPENMP(980)
[DEBUG:0@1.172] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: ONETBB (priority=1000)
[DEBUG:0@1.173] global plugin_parallel_wrapper.impl.hpp:219 getPluginCandidates core(parallel): ONETBB plugin's glob is 'libopencv_core_parallel_onetbb*.so', 1 location(s)
[DEBUG:0@1.176] global plugin_parallel_wrapper.impl.hpp:229 getPluginCandidates     - /usr/local/opencv44/lib: 0
[DEBUG:0@1.176] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 0 plugin(s) for ONETBB
[DEBUG:0@1.176] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: TBB (priority=990)
[DEBUG:0@1.177] global plugin_parallel_wrapper.impl.hpp:219 getPluginCandidates core(parallel): TBB plugin's glob is 'libopencv_core_parallel_tbb*.so', 1 location(s)
[DEBUG:0@1.179] global plugin_parallel_wrapper.impl.hpp:229 getPluginCandidates     - /usr/local/opencv44/lib: 0
[DEBUG:0@1.180] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 0 plugin(s) for TBB
[DEBUG:0@1.180] global parallel.cpp:65 createParallelForAPI core(parallel): trying backend: OPENMP (priority=980)
[DEBUG:0@1.180] global plugin_parallel_wrapper.impl.hpp:219 getPluginCandidates core(parallel): OPENMP plugin's glob is 'libopencv_core_parallel_openmp*.so', 1 location(s)
[DEBUG:0@1.182] global plugin_parallel_wrapper.impl.hpp:229 getPluginCandidates     - /usr/local/opencv44/lib: 0
[DEBUG:0@1.182] global plugin_parallel_wrapper.impl.hpp:233 getPluginCandidates Found 0 plugin(s) for OPENMP
[DEBUG:0@1.183] global parallel.cpp:93 createParallelForAPI core(parallel): fallback on builtin code
yolov8 = 1
x= 167.81
y= 376.606
w= 72.6373
h= 271.248
x= 167.792
y= 375.526
w= 72.5501
h= 270.116
x= 167.972
y= 375.102
w= 72.7924
h= 268.723
x= 167.951
y= 375.705
w= 72.6984
h= 269.759
x= 167.839
y= 373.922
w= 72.9471
h= 269.855
x= 168.025
y= 374.409
w= 72.7293
h= 270.39
x= 167.787
y= 374.763
w= 72.4588
h= 270.854
x= 23.3729
y= 421.572
w= 46.4561
h= 190.375
x= 23.1779
y= 421.486
w= 45.9497
h= 189.885
x= 167.861
y= 376.63
w= 72.9748
h= 266.303
x= 168.132
y= 376.905
w= 72.7592
h= 265.724
x= 167.837
y= 376.958
w= 72.2386
h= 266.638
x= 23.2711
y= 421.516
w= 46.2403
h= 190.333
x= 23.3293
y= 421.656
w= 46.2631
h= 190.686
x= 23.2795
y= 420.874
w= 46.2285
h= 191.039
x= 23.3943
y= 421.09
w= 46.3916
h= 191.826
x= 23.2661
y= 421.472
w= 46.2517
h= 191.094
x= 23.4039
y= 421.653
w= 46.4619
h= 191.458
x= 23.2874
y= 420.987
w= 46.3044
h= 192.041
x= 23.4548
y= 420.681
w= 46.6076
h= 192.748
x= 236.848
y= 286.737
w= 471.154
h= 301.516
x= 238.208
y= 286.811
w= 474.02
h= 300.861
x= 239.108
y= 286.802
w= 474.504
h= 300.796
x= 236.835
y= 286.723
w= 470.912
h= 301.118
x= 238.368
y= 287.157
w= 472.96
h= 301.713
x= 239.423
y= 287.073
w= 474.454
h= 301.332
x= 236.414
y= 286.922
w= 468.878
h= 303.659
x= 238.476
y= 287.072
w= 472.718
h= 303.736
x= 239.174
y= 287.194
w= 473.91
h= 303.299
x= 238.63
y= 287.406
w= 473.967
h= 303.518
x= 437.47
y= 378.019
w= 83.9446
h= 288.63
x= 437.948
y= 378.691
w= 83.6211
h= 289.386
x= 88.5217
y= 386.146
w= 116.07
h= 299.164
x= 88.4752
y= 386.59
w= 116.411
h= 299.438
x= 88.3197
y= 386.213
w= 116.248
h= 299.437
x= 437.838
y= 378.165
w= 83.5572
h= 289.434
x= 437.877
y= 378.49
w= 83.9373
h= 289.74
x= 88.6896
y= 386.691
w= 116.569
h= 299.52
x= 88.8605
y= 386.447
w= 116.491
h= 299.344
x= 88.4447
y= 386.505
w= 116.628
h= 299.305
x= 437.91
y= 378.028
w= 83.6533
h= 289.466
x= 437.813
y= 378.194
w= 84.1913
h= 289.956
x= 88.7102
y= 387.144
w= 117.173
h= 299.747
x= 88.7134
y= 386.239
w= 116.937
h= 299.668
x= 88.2784
y= 386.447
w= 116.777
h= 299.2
x= 437.966
y= 377.775
w= 83.7048
h= 288.954
x= 437.876
y= 377.937
w= 84.3022
h= 289.526
x= 88.7357
y= 386.285
w= 117.279
h= 300.284
x= 437.99
y= 377.839
w= 84.0185
h= 288.524
x= 438.098
y= 378.201
w= 84.0587
h= 288.138
Number of detections:5

The xywh recognized by GPU is all 0.
Using CPU is normal and will return the correct result.

Hi,

Do you mean the sample has no output even on other desktop GPU platforms?
If yes, could you check with the sample owner to see if it supports GPU mode?

Thanks.