Creating minimal SD card for headless inference

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

I’d like to create an SD card for the Nano that is as small as possible and only contains the packages needed to execute Tensor RT for inference. The Nano would not be connected to a monitor, so the complete Ubuntu desktop is not required.

I have started with removing almost all standard Ubuntu desktop packages for this (using apt remove) and everything that is related to it. Could you provide guidance around if/what of the XServer and Wayland packages are needed in order to keep the Jetson Nano’s HW acceleration working? Are there any other low level debian packages that are needed to ensure Jetson Nano’s HW acceleration stays working?

Here are the packages I have removed so far:

sudo apt remove ubuntu-desktop
sudo apt remove libreoffice-writer chromium-browser chromium* yelp unity thunderbird rhythmbox nautilus gnome-software
sudo apt remove ubuntu-artwork ubuntu-sounds ubuntu-wallpapers ubuntu-wallpapers-bionic
sudo apt remove vlc-data gdm
sudo apt remove unity-settings-daemon packagekit wamerican mysql-common libgdm1
sudo apt remove ubuntu-release-upgrader-gtk ubuntu-web-launchers

sudo apt-mark manual cuda-command-line-tools-10-0 cuda-compiler-10-0 cuda-cublas-dev-10-0 cuda-cufft-10-0 cuda-cufft-dev-10-0 cuda-cuobjdump-10-0 cuda-cupti-10-0 cuda-curand-10-0 cuda-curand-dev-10-0 cuda-cusolver-10-0 cuda-cusolver-dev-10-0 cuda-cusparse-10-0 cuda-cusparse-dev-10-0 cuda-gdb-10-0 cuda-gpu-library-advisor-10-0 cuda-libraries-dev-10-0 cuda-memcheck-10-0 cuda-misc-headers-10-0 cuda-nsight-compute-addon-l4t-10-0 cuda-nvcc-10-0 cuda-nvdisasm-10-0 cuda-nvgraph-10-0 cuda-nvgraph-dev-10-0 cuda-nvml-dev-10-0 cuda-nvprof-10-0 cuda-nvprune-10-0 cuda-nvrtc-10-0 cuda-nvrtc-dev-10-0 cuda-nvtx-10-0 cuda-tools-10-0 libxmu-dev libxmu-headers
sudo apt-mark manual python3-debconf python3-debian python3-yaml update-notifier-common

sudo apt remove cuda-toolkit-10-0 cuda-samples-10-0 cuda-documentation-10-0
sudo apt remove libqt5* qt5-gtk-platformtheme
sudo apt remove --purge libreoffice*
sudo apt remove bluez libboost-date-time1.65.1 libboost-filesystem1.65.1 libboost-iostreams1.65.1 libboost-locale1.65.1 
sudo apt remove remove snapd lightdm cups
gnome-applet* gnome-bluetooth gnome-desktop* gnome-sessio* gnome-user* gnome-shell-common gnome-control-center gnome-screenshot
sudo apt autoremove
2

What kind of hw acceleration do you need? Wayland and Xserver are needed if you need to render or leverage EGLimage.

3

I would only require HW accelaration as needed for AI inference in Tensor RT, i.e. running CUDA etc. Do we need to keep the XServer, X11 (or Wayland) running for this? I want to avoid the inference falling back to CPU only.

4

You can just switch off the x server with:

Temporary:

sudo systemctl isolate multi-user.target

And if you want to make the setting persistent so it will still be headless on reboot:

sudo systemctl set-default multi-user.target

There is no need to uninstall them to free up ram. Just turn them off and they won’t run. There is no need to uninstall them.

Tip. You can still access consoles via HDMI and switch between them with Ctrl+number. You can also still run X applications via ssh -X even if X isn’t running on the nano. X apps are clients that connect to a display server. The server doesn’t have to be on the same machine, however graphically demanding apps will likely not perform well remotely.

5

Thanks for your suggestions. I’m not looking for temporary turning off X or trying to run X remotely. Our system does not require any UI and we are trying to make an SD card image that has as few packages as possible (for size and maintenance reasons). We only want to keep the absolute minimum required to run Tensorflow RT with CUDA (i.e. hardware supported AI inference). I’d like to confirm what components are needed to achieve that. I heard from other Jetson users that X might be required and removing it might disable CUDA or related things that Tensorflow RT is using.

6

I can now confirm that the xserver is not needed. CUDA and tensorflow are still working after removing the apt packages for X. You just have to make sure the nvidia cuda packages are not automatically removed. For this, run

sudo apt-mark manual cuda-command-line-tools-10-0 cuda-compiler-10-0 cuda-cufft-10-0 cuda-cufft-dev-10-0 cuda-cuobjdump-10-0 cuda-cupti-10-0 cuda-curand-10-0 cuda-curand-dev-10-0 cuda-cusolver-10-0 cuda-cusolver-dev-10-0 cuda-cusparse-10-0 cuda-cusparse-dev-10-0 cuda-gdb-10-0 cuda-gpu-library-advisor-10-0 cuda-libraries-dev-10-0 cuda-memcheck-10-0 cuda-misc-headers-10-0 cuda-nsight-compute-addon-l4t-10-0 cuda-nvcc-10-0 cuda-nvdisasm-10-0 cuda-nvgraph-10-0 cuda-nvgraph-dev-10-0 cuda-nvml-dev-10-0 cuda-nvprof-10-0 cuda-nvprune-10-0 cuda-nvrtc-10-0 cuda-nvrtc-dev-10-0 cuda-nvtx-10-0 cuda-tools-10-0 cuda-documentation-10-0 cuda-samples-10-0 cuda-toolkit-10-0

Afterwards, X and associated packages can be removed with

sudo apt remove x11-common x11proto-* mesa-* libgl1* libglapi-mesa libgles* libglu1-mesa libglvnd* libglx* libx11-* libwayland-*