Degradation of performance when running AI computation on NVIDIA Drive AGX

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info:
Software Version
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.4.0.7363
other

Host Machine Version
native Ubuntu 18.04
other

We’re having some questions regarding optimizing applications running on NVIDIA Drive with Drive SDK - Linux OS (Ubuntu 18.04). Currently our AI software is facing serious performance degradation when running on NVIDIA Drive, it drops ~50% speed compared to running on Laptop ROG STRIX G15 G512L-WAZ114T or even on Jetson AGX (Pytorch 1.6 backbone resnet34 inference time: 25ms on Jetson AGX, 44ms on Drive AGX).

We’d followed this guide to set up our NVIDIA Drive board: Install DRIVE with SDK Manager :: NVIDIA DRIVE AGX System Installation and Setup

So I’d like to consult you the following questions::

  1. What is the recommended ways to benchmark, optimize the performance of AI applications on NVIDIA Drive? Could you give us some guidelines about Optimization on NVIDIA Drive, especially for Linux OS?
  2. We realize that there’s Hypervisor mode running when installing Linux OS to the board via NVIDIA Drive SDK Manager as the above link. So could you guide us on how to turn off the Hypervisor and how to install Linux OS as bare-metal?

I’m looking forward to your responses. Thanks in advance.

3

Dear @v.tripc2 ,
Could you share details about your use case

4

Sorry, It’s my typo. I edited the question. Please help to check it @SivaRamaKrishnaNV . Thanks.

5

Dear @v.tripc2 ,
We recommend using TensorRT model on DRIVE for good performance. Also, DRIVE SW 10 has TRT 5.x and the latest DRIVE release has TRT 6.x. could you please upgrade to latest DRIVE OS 5.2 + DW 3.5 release and see if it improves.

6

Dear @SivaRamaKrishnaNV,
We’ve been trying to convert our AI algos to use TensorRT.
Because of the development process reason, we could only use the Linux OS for now. So I’d like to ask:

  1. Do you have any guidelines to benchmark, optimize the performance of AI applications on NVIDIA Drive (Linux)?
  2. How about the Hypervisor mode, do you know how to turn it off? or could you guide us how to install Linux OS (without Hypervisor) onto NVIDIA Drive AGX?
7

Dear @v.tripc2 ,
Q1. We do not officially support any DL framework installation/setup on DRIVE AGX platform. As I said the recommended approach is to covert DL model → ONNX → TRT model and perform inference. Using TensorRT helps to make use of HW accelerators like GPU, DLA on Xavier SoC .
Q2 . It is not possible to just install bare linux OS using sdkmanager.

8

Thanks @SivaRamaKrishnaNV for the useful information,

→ We’re converting the model as your suggestion, we’ll revert back to you with the result ASAP.

→ How about not using the sdkmanager, is it possible to do that (installing bare Linux OS)?
→ BTW, do you have any documentation regarding the internal mechanism of Hypervisor? we also want to know how the Hypervisor affects applications running on the Linux OS. In other words, we’d like to know the maximum computing capability of applications running in Hypervisor mode.

9

Dear @v.tripc2 ,

  1. You need use sdkmanager only to get latest DRIVE SW release on the target.
  2. Could you please check DRIVE OS Linux and see if it helps ?
10

Thanks a lot for the hints @SivaRamaKrishnaNV, we’ve been trying to do the following things:

  • Converting our models to using TensorRT
  • It seems we’d set up our board using NVIDIA DRIVE™ Software for DRIVE AGX, we’ll try to install the NVIDIA DRIVE™ OS for DRIVE AGX to see if it improves the performance.

We’ll get back to you as soon as we get the result.