Jetson AGX Xavier with RT kernel and Realsense camera modules

Our goal was to setup a Jetson AGX Xavier to run a vision-based grasping application. To control the robot, we needed a preemptive realtime kernel and for object detection we used a Intel Realsense D435i camera. To get both running we took the following steps:

  1. We started with having a look at which does not support Xavier.
  2. There are discussions on this topic in the forum (
  3. Inside the kernel source a script can be found which applies the rt patch. But there is no official documentation or support about it, so it might be an experimental setup.
  4. Based on these sources we applied the rt patch to L4T. We did not manage to patch the running system, instead we patched the kernel before flashing it to the board.
  5. Follow the instructions in the section "Kernel Customization" of the L4T documentation but before "Building the NVIDIA Kernel" the RT patch was applied: 1. modify /kernel/source/scripts/ to use tegra_defconfig instead of tegra_gnu_linux_defconfig 2. apply the patch by running /kernel/source$ ./scripts/ apply-patches
  6. Continue with "Building the NVIDIA Kernel"
  7. There is an unofficial hack for using Intel Realsense Cameras on Jetson devices at which we used. The provided scripts needed to be adapted to the rt-patched kernel and the package sources had some issues, so the sources files had to be downloaded manually to be sure to get the right ones. We patched the install scripts from to get things running.
  8. With the described setup, we still had to reduce the cameras frame-rate to about 6Hz to avoid occasional loss of connection to the device. This might be a problem of the USB-bus rather than the kernel setup though..

Note: With this setup we experienced that using the script ~/ (to run at maximum CPU performance) resulted in stopped fans which caused the Xavier to quickly overheat. We manually removed the responsible line from the shell-script to fix it.


RT kernel doesn’t support Jetson yet.
This feature is in our future plan but is not ready yet.