How to execute external applications on GPU?

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
[*] DRIVE OS 6.0.10.0
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
[*] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
[*] DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
2.1.0
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
[*] native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Issue Description
I have an external python based application which actually does the Vision processing like reading frames and performing the object detection and annotation. I want to run this application on GPU. Can you please let me know how to execute any custom application on GPU?

3

Dear @PA_GN ,
You need to implement the functionality as CUDA kernel to run on GPU.

4

Hi @SivaRamaKrishnaNV,

Can you please give more information on this? Is there any references or documents available?

5

Hi @SivaRamaKrishnaNV,

Please share more details.

6

Dear @PA_GN,

If it is just python code which runs sequentially on CPU, there is no way to offload to GPU directly. As clarified earlier, You need to implement the function as CUDA kernel to make it run on GPU.
We have couple of CUDA samples at /usr/local/cuda-11.4/samples for your reference. You can take look at CUDA C++ Programming Guide for details on CUDA programming.

7

Hi @SivaRamaKrishnaNV,

Any specific reason why explicit applications like Python or C/C++ wont be able to access GPU as we have seen on x86 based machines which are having GPU configured; on these systems using below command the python application will run on GPU which does not happen in case of Orin.

python detect.py --source <your_image_or_video_source> --device 0 --weights <path_to_yolov8_weights>

8

Dear @PA_GN,
I am not sure why your application could not run on GPU. but, We have CUDA c++ samples and TRT python samples on DRIVE OS which can run on GPU.

9

Hi @SivaRamaKrishnaNV,

Can you please share the path for the python samples and details to validate their execution on GPU?

10

Dear @PA_GN ,
Please see /usr/src/tensorrt/samples/python for TRT python samples.
Please check ReadMe file for each sample for requirements and guidance. In case of any issue, please file new topic with all details.

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.