Custom Rootfs Permission/Performance issues

Hello,

I have created a custom root file system form the nano using ubuntu-base 18.04.2 as a starting point.

the image I created is based on Lubuntu/Lxqt

  • Display manager is SDDM
  • Window manger is Openbox
  • Compositor is Compton/Compiz

after installing all the required packages through a chroot enviroment I use the apply_binaries.sh script to include the Nvidia drivers and init scripts and I use the create-jetson-nano-sd-card script to produce an Sdcard image.

The system seems to boot fine and I am presented with the lxqt desktop environment after logging in however there seems to be some issues with permission as I am unable to launch any applications that use the nvidia drivers.

Any application that uses OpenGL need to be launched with root privileges to function including the compositors I mentioned above. if they are not launched with root privilages I get a Segmentation fault (core dumped ) error.

I have added the display manager to the Video group however this doesnt seem to make a difference as far as applications being able to interact with the display drivers.

When launching Compiz with root the desktop performance is reduced significantly even though the settings I’m using are the same as the ones used in the L4T image that is provided for the nano.

When i attempt to use compton with the GLX render setting the screen goes crazy and spams “glx_render_(): Missing texture” in the terminal.

[Edit] (The isssue with Compton seems to have been due to the nvidia logo wallpaper being rendered behind the regular walpaper compton seems to be working now however overall performance is lacking in comparison to L4T as well.)

So it’s clear that I have some permission issues and the xserver isn’t able to make full use of the display drivers. when I do manage to get compiz working the performance is terrible in comparison to the normal L4T image even though I believe “Lubuntu/Lxqt” is more lightweight than Gnome/Unity.

Can anyone help me get things straightened out?

ill attach some logs and glx info.

Thank you!!

Xorg.txt (9.17 KB)
system.txt (149 KB)
glxinfo.txt (1.03 KB)

jetson@tegra-ubuntu:~$ sha1sum -c /etc/nv_tegra_release
/usr/lib/aarch64-linux-gnu/tegra/libnvosd.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_camera.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_image.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvomx.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_force.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnveventlib.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmedia.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_utils.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvexif.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtx_helper.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libsensors.l4t.no_fusion.nvs.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvscf.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libsensors.hal-client.nvs.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvfnetstorehdfx.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmm_parser.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvrm.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmm_contentpipe.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_gpucompute.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_tbc.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvos.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtnr.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_graphics.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libsensors_hal.nvs.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvimp.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvfnet.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvphs.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvavp.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_video.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvfnetstoredefog.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_il.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvodm_imager.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtvmr.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtracebuf.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvidia-egl-wayland.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvdc.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_boot.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_generic.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libtegrav4l2.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmm.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvapputil.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcameratools.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcam_imageencoder.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnveglstream_camconsumer.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmm_utils.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvomxilclient.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvwinsys.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvargus.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_spincircle.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvphsd.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnveglstreamproducer.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvll.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvrm_graphics.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcolorutil.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtestresults.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcamv4l2.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcamerautils.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcamlog.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvparser.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvddk_vic.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_ui.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite.so: OK
/usr/lib/aarch64-linux-gnu/libv4lconvert.so: OK
/usr/lib/aarch64-linux-gnu/libv4l/plugins/libv4l2_nvvidconv.so: OK
/usr/lib/aarch64-linux-gnu/libv4l/plugins/libv4l2_nvvideocodec.so: OK
/usr/lib/xorg/modules/drivers/nvidia_drv.so: OK
/usr/lib/xorg/modules/extensions/libglxserver_nvidia.so: OK
jetson@tegra-ubuntu:~$

I see that the rules for the GPU components are being set by a udev script 99-tegra-device.rules

Should I need to change the permissions in the file?

or do i simply add myself to the Video group

is that normally necessary?

I appreciate any input.

Chris.

This should help you (from the default rootfs):

adminuser@fixed-nano:~/Pictures$ groups
adminuser adm cdrom sudo audio dip video plugdev lpadmin gdm sambashare
adminuser@fixed-nano:~/Pictures$ sudo adduser someuser
[sudo] password for adminuser: 
Adding user `someuser' ...
Adding new group `someuser' (1001) ...
Adding new user `someuser' (1001) with group `someuser' ...
Creating home directory `/home/someuser' ...
Copying files from `/etc/skel' ...
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully
Changing the user information for someuser
Enter the new value, or press ENTER for the default
	Full Name []: some user
	Room Number []: 
	Work Phone []: 
	Home Phone []: 
	Other []: 
Is the information correct? [Y/n] y
Adding new user `someuser' to extra groups ...
Adding user `someuser' to group `audio' ...
Adding user `someuser' to group `video' ...
Adding user `someuser' to group `gdm' ...
adminuser@fixed-nano:~/Pictures$

relavant lines from /etc/group for the GIDs:

audio:x:29:pulse,adminuser,someuser
video:x:44:adminuser,gdm,someuser
gdm:x:124:adminuser,someuser

from /etc/passwd for the display manager:

gdm:x:120:124:Gnome Display Manager:/var/lib/gdm3:/bin/false

So Nvidia has the display manager in the video group. I suspect there is a good reason for that related to hw acceleration but you said you could log in so ¯_(ツ)_/¯

Maybe it’s not strictly required but only affects perforamce. Might I suggest, however, that instead of creating a new linux distro, it may be easier to modify the rootfs nvidia provides.

If you want to speed up the ux on the stock image, i might suggest:

sudo apt update && sudo apt install gnome-tweaks -y && gnome-tweaks

Then simply switch off animations. I do this on my workstation even since it’s quicker to change tasks and frankly less janky on an IGP driving two 4k screens.

I am not sure where you would change this on the rootfs nvidia provides, but the configuration template has to be somewhere.

edit, so i dug this up:

$ gsettings set org.gnome.desktop.interface enable-animations false

is the gsettings command to turn the animations off.

And here is how you can set it systemwide as the default, but i’ve never tested (it’s adding a file to a .d folder in /etc/ on nvidia’s provided rootfs):

https://unix.stackexchange.com/questions/289640/how-to-create-a-default-system-wide-dconf-setting-starting-from-just-created-ad

Yes, you would have to add your user to group video.

sudo usermod -a -G video your_user

This is usual with L4T.

@Honey_Patouceu: Is it necessary to do this for the display manager user on a custom rootfs as well (gdm, lightdm)?

@OP: Do you know what display manager your rootfs uses?

If it’s gdm, you should probably be able to:

sudo usermod -a -G video gdm

@mdegans

On Xavier R31.1, Ubuntu18 using gdm, I indeed see gdm in video group:

cat /etc/group | grep video
video:x:44:ubuntu,nvidia,<b>gdm</b>

For lightdm, it may not be required. On TX2 R28.2, Ubuntu16 using lightdm I only see standard users:

cat /etc/group | grep video
video:x:44:ubuntu,nvidia

Thanks for the info , so in my lubuntu/lxqt build the display manager is SDDM and adding SDDM to the video group doesn’t seem to do anything in regards to being able to utilize the OpenGL drivers. However once I add my main user account to the video group I’m able to launch applications that use opengl no problem…including the compositors.

I’ve always modified the apply_binaries script and replaced every instance of the word gdm3 with whichever display manager I plan on using so.

I just don’t recall ever needing to include my admin
account in the video group in the past. Maybe it’s just the way Nvidia has the driver’s setup.

There’s so many little init scripts and whatnot it’s difficult to figure out what’s going on behind the scenes especially considering my limited knowledge of Linux.

I just created a new xubuntu-core based image and the the performance seems to be a bit better however the the compositor built into xfwm4 doesn’t seem to work with the Nvidia drivers so I’m forced to use Compton or compiz.

Once I get the kinks worked out maybe I’ll post the image for others to try out …it’s definately alot more lightweight than the l4t image.

I’ll include the local CUDA repos and make a script that will install the debs if one chooses to that way the image should be under 5 gigs.

Systemd handles most startup things on Ubuntu in lieu of init.d scripts. Now you have some text files and symlinks in /etc/systemd/system with some configuration in it defining the script, user it’s run as, etc.

You enable or disable services with the service command sysctl. Reading systemd’s full manual is not really advisable for the sane. Suggest instead, trying to find good working examples and modifying those.

Glad it’s working for you!

As a side note, how difficult do you guys think it would be to port chrome os to the nano? I assume that would involve some pretty serious kernel manipulation…

Thanks again.
Chris.

I don’t know if that will work (See below), suggest not modifying apply_binaries.sh

$ fgrep gdm apply_binaries.sh 
# If default target does not exist and if rootfs contains gdm, set default to oem-config target
    [ -d "${LDK_ROOTFS_DIR}/etc/gdm3/" ];then
	extra_groups="EXTRA_GROUPS=\"audio video gdm\""
if [ -e "${LDK_ROOTFS_DIR}/etc/gdm3/custom.conf" ]; then
	sed -i "/WaylandEnable=false/ s/^#//" "${LDK_ROOTFS_DIR}/etc/gdm3/custom.conf"

Well if I’m not installing gdm there isn’t really a reason to add new users to a gdm group so by replacing that with whatever display manager I plan on using I’m creating a situation where anytime a new user is created they will be added to the display manager group, the audio group and the video group. … I think lol

Bottom line the user account needs to be added to the video group and the display manager group and the display manager needs to be a member of the video group.

Does anyone know where I could upload my image to make it available for ppl to download once it’s complete?

Github.com or sourceforge.net.