Orin 36.3 with Preempt_RT patch

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

Hi Nvidia!

I’m trying to install Preempt_RT patch on Orin, for better model inference speed.

With the official docs of 36.3

Screenshot from 2024-09-04 19-57-22
Screenshot from 2024-09-04 19-57-22862×582 35.2 KB

It shows the PREMPT_RT is enabled in the pre-built kernel Debian package:

I have asked nvidia engineer for some advises, with the official kernel customization docs
https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/SD/Kernel/KernelCustomization.html

image
image862×511 49.4 KB

The engineer told me the preempt_rt patch should be active manually, as the step2 in Building the Jetson Linux Kernel shows, just like the jetson_clocks?

I found this github about install Preempt_RT on jetson board

But this guide just tested on Xavier developer kit and Jetson Nano, and is recommended based on Ubuntu 20.04.6 LTS, while Jetson Linux version is 36.3.0. which Ubuntu system is different from l4t36.3 official ubuntu version 22.04.
So I’m not sure it will work or will broke the system.

I want to know which method I should take to active Preempt_RT?
Just follow the github shows?
Or follow the official kernel customization docs? If the kernel customization docs work, how much step should I take? just till step2 in Building the Jetson Linux Kernel, or finish the Building the Jetson Linux Kernel block? or more steps?

Thank you!

4

Hi,
There is debian package for AGX Orin developer kit:

Software Packages and the Update Mechanism — NVIDIA Jetson Linux Developer Guide 1 documentation

If you use your own custom board, you would need to manually enable it. Please check

Kernel Customization — NVIDIA Jetson Linux Developer Guide 1 documentation

5

Thank you for your answer, DaneLLL.

I wonder what is the different between the Real-Time Kernel Using OTA Update and Building the Jetson Linux Kernel

If I want to get the maximum performance of official Orin development kit, only update real-time kernel is enough? Or I still need to build Building the Jetson Linux Kernel?

6

Hi,
If you don’t need multiple processes in your use-case, you may try RT kernel.

Please execute sudo jetson_clocks to achieve maximum throughput on either RT kernel or non RT kernel.

7

Hi DaneLLL!

We may have the requirement of multiple precesses. So……I need to make the Preempt_RT patch, Right?

Or could you tell me, If I did the Preempt_RT patch, jetson36.3 system can inference normally? Will the Preempt_RT patch affect the llm/vlm model inference speed and result?

8

Hi,
Please set up both environments and give it a try. Ideally it should have similar throughput on the two setups, but it is case-by-case. Would suggest set up the real environments and do comparison.

9

Hi DaneLLL,

Thank you for your help!

I heard that on the RTX 4090 GPU, If made the Preempt_RT patch, The system can not find the GPU board. So I am afraid that the Preempt_RT patch will affect the Orin development kit’s AI performance, even broken down the whole AI inference system.