How to successfully update Jetson TK1 to Ubuntu 16.04

I managed to update a TK1 (DJI Manifold) to 16.04 following this procedure, but I have no power to my USB devices after the update.

lsusb is empty

Any ideas for follow-up steps?

Below is my grep of dmesg.

ubuntu@tegra-ubuntu:~$ dmesg | grep usb
[    0.000000] Kernel command line: console=ttyS0,115200n8 console=tty1 no_console_suspend=1 lp0_vec=2064@0xf46ff000 mem=2015M@2048M memtype=255 ddr_die=2048M@2048M section=256M pmuboard=0x0177:0x0000:0x02:0x43:0x00 tsec=32M@3913M otf_key=c75e5bb91eb3bd947560357b64422f85 usbcore.old_scheme_first=1 core_edp_mv=1150 core_edp_ma=4000 tegraid=40.1.1.0.0 debug_uartport=lsport,3 power_supply=Adapter audio_codec=rt5640 modem_id=0 android.kerneltype=normal fbcon=map:1 commchip_id=0 usb_port_owner_info=2 lane_owner_info=6 emc_max_dvfs=0 touch_id=0@0 board_info=0x0177:0x0000:0x02:0x43:0x00 root=/dev/mmcblk0p1 rw rootwait tegraboot=sdmmc gpt
[    0.531983] usbcore: registered new interface driver usbfs
[    0.532047] usbcore: registered new interface driver hub
[    0.532161] usbcore: registered new device driver usb
[    0.660063] usb0-vbus: 5000 mV 
[    0.660361] usb1-usb2-vbus: 5000 mV 
[    4.145021] usbcore: registered new interface driver asix
[    4.145065] usbcore: registered new interface driver ax88179_178a
[    4.145111] usbcore: registered new interface driver cdc_ether
[    4.145166] usbcore: registered new interface driver smsc95xx
[    4.145206] usbcore: registered new interface driver cdc_subset
[    4.145274] usbcore: registered new interface driver cdc_ncm
[    4.145781] tegra_xusb_read_usb_calib: usb_calib0 = 0x02858389
[    4.146272] usbcore: registered new interface driver usb-storage
[    4.146384] usbcore: registered new interface driver usbserial
[    4.146420] usbcore: registered new interface driver option
[    4.146453] usbserial: USB Serial support registered for GSM modem (1-port)
[    4.146487] usbcore: registered new interface driver pl2303
[    4.146517] usbserial: USB Serial support registered for pl2303
[    4.152139] tegra-udc tegra-udc.0: usb_bat_chg regulator not registered: USB charging will not be enabled
[    4.163950] usbcore: registered new interface driver uvcvideo
[    5.204885] usbcore: registered new interface driver usbhid
[    5.215388] usbhid: USB HID core driver
[    5.325541] usbcore: registered new interface driver snd-usb-audio
[    8.093459] usb1-usb2-vbus: incomplete constraints, leaving on
[    8.102815] usb0-vbus: incomplete constraints, leaving on
[   11.231416] tegra-xhci tegra-xhci: failed to init firmware from filesystem: tegra_xusb_firmware

A custom board requires a custom device tree. I don’t know if the update overwrote any of the device tree settings, but my guess is that the Ubuntu mechanism doesn’t touch the tree. On the other hand, the USB driver might have been replaced with a non-Tegra version and incapable of functioning, but that is speculation.

A newer kernel might require the device tree to change. In that case you are out of luck unless you know what the hardware differences are and kernel version differences.

FYI, when flashing a Jetson TK1 dev kit on command line you’d use the driver package plus sample rootfs. The sample rootfs in this case is just Ubuntu 14.04 without the NVIDIA-specific drivers. When doing the command line flash it would go like this…see URL and adjust:
https://devtalk.nvidia.com/default/topic/982779/jetson-tk1/how-to-perform-a-correct-fresh-ubuntu-install-on-jetson-tk1/post/5040584/#5040584

In particular, the part which is of interest for you is what the “sudo ./apply_binaries.sh” step does. This unpacks the NVIDIA-specific hardware accelerated drivers and libraries into the “rootfs/” subdirectory, and then the flash creates the image from that “rootfs/” directory and a few bootloader edits.

If you get the “driver” package (L4T downloads are free, though you might need to create a second login…the version to get would depend what was previously on the system, and I can’t even guess what is on a DJI Manifold), and don’t unpack the sample rootfs, but do run the “apply_binaries.sh” step, then the “rootfs/” subdirectory will contain the unpacked binaries. Or you can create an empty temp directory somewhere, and name that as the unpack location via the “-r ” argument. Example:

sudo -s
mkdir /tmp/rootfs
./apply_binaries.sh -r /tmp/rootfs
# explore or copy or move the tree in "/tmp/rootfs/".
exit

When a Jetson has L4T on it you can run this command to check if all of those files are valid:

sha1sum -c /etc/nv_tegra_release

Does this command work for you? Does it show all files ok? If not, then perhaps unpacking those files from “/” of your TK1 would put a file back in place which is missing. I won’t even guess though at which files might be missing, but you could report here if anything specific is gone. There is no official support for this method of upgrade, nor for that version of Ubuntu, so there is a lot of guessing to live with.

I can see your usb_port_owner_info=2 do you have usb 3.0 enabled in odmdata in jetson-tk1.conf?

from jetson-tk1.conf

# ODMDATA for USB2.0 configuration on USB port(J1C2 connector) = 0x6009C000
# ODMDATA for USB3.0 configuration on USB port(J1C2 connector) = 0x6209C000
ODMDATA=0x6009C000;

Board command line:
usb_port_owner_info=0 = usb 2.0
usb_port_owner_info=2 = usb 3.0

Also im sure the DJI uses a custom dtb make sure your extlinux.conf is correct:

Jetson TK1 reads:

LABEL primary
MENU LABEL primary kernel
LINUX /boot/zImage
FDT /boot/tegra124-jetson_tk1-pm375-000-c00-00.dtb

Doing this upgrade now on my TK1, sitting watching the % creep higher and not hang :)

100% downloaded... Check
Upgrading.....
Unpacking....
Processing....

So sure it's going to hang after I've been screwing around with python upgrades, jetpack upgrades, yolo, cuda, you name it....

Still going :)

This is so tense :D :D

It's either going to hang, or my battery is going to run out (i'm running the tk1 on a 12v battery)

Still Not hanging....

Unpacking...
Preparing....
Reading....
Removing.....

Coffee...

Do I want to restart some services....Hmmm Ok

installing... 35%
.... 50%
... Unpacking fonts
Ouch.

Hung.

Well, I think the moral of that story is not to try to upgrade a 16GB mmc system to 16.04 when you've already filled the 14.04 system with a ton of experiments.

Options now:
a) Chuck it in the bin
b) Reinstall 14.04

I only wanted to see if I could install NCSDK on the TK1 with 16.04, oh well. 

Perhaps Nvidia can collaborate with Intel or Perhaps release it's own VPU stick or pcie board, or something cool like that.

Oh wait We're still going...

65%

Go TK1! Go!

More coffee....

70%

Warp speed Tk1!

80%!

Oh i'm sooooo excited right now

90% OMG!

98%

I'm going to vomit...
This is too much...

Searching for obsolete software....

Removing..

Ok now i'm getting bored.

Oh it worked!

16.04 Xenial
Seems I didn’t have to do ‘sudo apt-get install gnome-session-flashback’ either
Chromium works better on 16.04 than 14.04

I’m tired now, but I will add further progress later. Thanks for letting me know 16.04 works! (Even with a semi-full TK1)

Next Day update:

  1. I tried using Auto-Login at startup, as seems it the TK1 might have small issue with waking after sleep at login but no big deal if you just type the password in black screen it comes back. Using auto Login seem to affect chromium keychain request and problems with window.

  2. Chromium is a much better experience under 16.04 than 14.04:
    1080p videos play without lag unlike only 720p under 14.04
    Youtube plays just fine
    Can also now install Chromium extensions such as Ublock Origin, couldn’t do that before
    Many other sites which did not run before due to flash or plugin problems work now
    No Spotify web player just now
    3.8Gb free space XD

  3. I think i’m going to have to dump Cuda Demo folder and some other things to free up space so I can attempt to install NCSDK

Will try to update more advantages of 16.04 TK1

How Can I Upgrade Kernel?

If you don’t follow the instructions exactly and mess up, ending with this:

apt: relocation error: /usr/lib/arm-linux-gnueabihf/libapt-pkg.so.5.0: symbol _ZTVNSt7__cxx1119basic_istringstreamIcSt11char_traitsIcESaIcEEE version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference

Any hope of saving this or should I just flash it again with L4T?

Do you have to install JetPack after upgrading to Ubuntu 16.04?

You can certainly install Jetpack without issue but is is not absolutely needed unless you need the cuda compiler among other things included in Jetpack.

Thanks for the reply!

Is it possible to install CUDA 6.5 on the TK1 after upgrading to Ubuntu 16.04?

I would also like to know if it is possible to install CUDA 6.5 after updating to 16.04. My nvcc compiler broke after the update and I cannot work on my code now.

Hi ,
Did you face any shutdown issues unexpectedly after the update?

Don’t go past that too far. 18 seemed ok, then I went to 20 and light dm started failing and my board won’t boot. And reflashing the emmc needs an old Linux and doesn’t seem to work on the VM. I might have to actually install Ubuntu 16 to make it work again…

Can you please share how to upgrade to ubuntu 18.04

It does not appear that this is possible. I’d love to bring my TK1’s up to 16.04 or even 18, but the problem I’m having is that most of the repos have all removed armhf architecture.

Good evening everyone,

I’m an owner of a ChromeBook that also has an NVIDIA Tegra K1 SoC like the Jetson TK1. I’ve recently have taken a look at getting Ubuntu 22.04 LTS working on the device with Ubuntu Xenial Xorg LTS packages. It runs the old ChromeOS kernel, so it also has the ‘nvpgu’ driver just as the L4T 21.8 Linux 3.10 kernel does. I can most certainly say that it does work and it runs quite well.

A post in here suggests that 18.04 LTS Xorg packages seems to also work, potentially 20.04 as well, but also perhaps not (according to my understanding, testing, and research). I could give that I go, I know for a fact that the Xenial Xorg does and I do know for a fact that starting with some Xorg version do the NVIDIA blobs no longer work. There was a case on my end, a version of Xorg (that was too new) I tried simply segfaulted and not even the NVIDIA logo appeared; that was on Debian bullseye.

I noticed that for some reason, a change in glibc (I’m guessing) makes the linker configs (that point to the NVIDIA blobs) no longer work correctly, at least, it appears that way to me. Manually adding them to ‘/etc/ld.so.conf.d/arm-linux-gnueabihf.conf’ and then running ‘ldconfig’ with root perms (such as with ‘sudo’) will fix this.

This is what it looks like for me on that ChromeBook:

# Multiarch support
/usr/lib/arm-linux-gnueaibhf/tegra
/usr/lib/arm-linux-gnueaibhf/tegra-egl
/usr/local/lib/arm-linux-gnueabihf
/lib/arm-linux-gnueabihf

Most people here will be using NVIDIA’s rootfs. Ultimately, the laptop’s Ubuntu rootfs originated from an Ubuntu Xenial LTS installation that has been upgraded to Jammy. Moreover, I had to patch several deb files to prevent dependency conflicts (which I plan to share). For some reason upgrading from Xenial, to Bionic, to Focal, and then Jammy was the only way I was able to retain the older Xorg packages and have a working modern Ubuntu system as any other way caused major issues. I also had to have a pin.pref file (which I’ll also share here). I don’t own a Jetson TK1, but I plan to experiment with a fresh copy of NVIDIA’s L4T 21.8 chroot though as I noticed my method of setting this up appears to be sensitive in a way, which is why I’m going to take a more in-depth look at NVIDIA’s Ubuntu Trust Tahr rootfs. It took many hours of trial and error to get my ChromeBook in its current state. Also, for some reason, the ‘apply_binaries.sh’ file has corrupted several of my rootfs’s and sometimes it doesn’t. I don’t really know why that is either.

From my experience, the Unity desktop was usable yet also problematic (it seems like there was some change that made it less usable with the blobs). The laptop runs the Mate desktop works and Mate runs wonderfully. For Mate, I needed two dummy packages to stop it from running into dependency conflicts. I have yet to try KDE. Gnome still doesn’t run without some problem/problems, rendering it to be unusable, in fact, if it ever ran on Tegra K1 GPUs, maybe it was in during the time of Ubuntu Trusty Tahr as it didn’t even work in Xenial when I was experimenting with an Xenial install that I had made before managing to get Jammy working.

It would be more ideal to get a newer kernel running. I have absolutely no kernel module porting experience or even kernel modification experience for that matter, but I’ve taken a look at the nvgpu kernel modules and so far, it seems like the oldest, most currently supported in some form Linux kernel that could work with minimal changes is 4.4 by Civil Infrastructure Platform. After a bit of modifying with the sources, I’ve managed to get some of the kernel modules to build, but much work is left to be done and I have absolutely no idea how to get passed some of the errors that I’ve been receiving neither if some of the work I’ve done that has gotten it to build partially would actually work in real use.

If anyone is wondering about where I’m getting stuck at, this is where:

drivers/gpu/nvgpu/gk20a/gk20a.c: In function 'gk20a_pm_initialise_domain':
drivers/gpu/nvgpu/gk20a/gk20a.c:1272:3: error: implicit declaration of function 'pm_genpd_set_poweroff_delay' [-Werror=implicit-function-declaration]
 1272 |   pm_genpd_set_poweroff_delay(domain, platform->railgate_delay);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/gk20a.c: In function 'gk20a_secure_page_alloc':
drivers/gpu/nvgpu/gk20a/gk20a.c:1315:3: error: implicit declaration of function 'tegra_periph_reset_assert' [-Werror=implicit-function-declaration]
 1315 |   tegra_periph_reset_assert(platform->clk[0]);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/pmu_gk20a.c: In function 'gk20a_init_pmu_setup_sw':
drivers/gpu/nvgpu/gk20a/pmu_gk20a.c:1588:15: error: 'DMA_ATTR_READ_ONLY' undeclared (first use in this function)
 1588 |  dma_set_attr(DMA_ATTR_READ_ONLY, &attrs);
      |               ^~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/pmu_gk20a.c:1588:15: note: each undeclared identifier is reported only once for each function it appears in
drivers/gpu/nvgpu/gk20a/gk20a.c:1318:3: error: implicit declaration of function 'tegra_periph_reset_deassert' [-Werror=implicit-function-declaration]
 1318 |   tegra_periph_reset_deassert(platform->clk[0]);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/pmu_gk20a.c: At top level:
drivers/gpu/nvgpu/gk20a/pmu_gk20a.c:3530:5: error: "CONFIG_DEBUG_FS" is not defined, evaluates to 0 [-Werror=undef]
 3530 | #if CONFIG_DEBUG_FS
      |     ^~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c: In function 'gk20a_init_clk_setup_sw':
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:483:8: error: implicit declaration of function 'tegra_dvfs_get_freqs' [-Werror=implicit-function-declaration]
  483 |  err = tegra_dvfs_get_freqs(clk_get_parent(clk->tegra_clk),
      |        ^~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c: At top level:
drivers/gpu/nvgpu/gk20a/gk20a.c: In function 'gk20a_remove':
drivers/gpu/nvgpu/gk20a/gk20a.c:1513:2: error: implicit declaration of function 'nvhost_module_disable_clk'; did you mean 'nvhost_module_idle_ext'? [-Werror=implicit-function-declaration]
 1513 |  nvhost_module_disable_clk(&dev->dev);
      |  ^~~~~~~~~~~~~~~~~~~~~~~~~
      |  nvhost_module_idle_ext
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:638:15: error: variable 'gk20a_clk_export_ops' has initializer but incomplete type
  638 | static struct tegra_clk_export_ops gk20a_clk_export_ops = {
      |               ^~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:639:3: error: 'struct tegra_clk_export_ops' has no member named 'init'
  639 |  .init = gk20a_clk_export_init,
      |   ^~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:639:10: error: excess elements in struct initializer [-Werror]
  639 |  .init = gk20a_clk_export_init,
      |          ^~~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:639:10: note: (near initialization for 'gk20a_clk_export_ops')
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:640:3: error: 'struct tegra_clk_export_ops' has no member named 'enable'
  640 |  .enable = gk20a_clk_export_enable,
      |   ^~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:640:12: error: excess elements in struct initializer [-Werror]
  640 |  .enable = gk20a_clk_export_enable,
      |            ^~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:640:12: note: (near initialization for 'gk20a_clk_export_ops')
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:641:3: error: 'struct tegra_clk_export_ops' has no member named 'disable'
  641 |  .disable = gk20a_clk_export_disable,
      |   ^~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:641:13: error: excess elements in struct initializer [-Werror]
  641 |  .disable = gk20a_clk_export_disable,
      |             ^~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:641:13: note: (near initialization for 'gk20a_clk_export_ops')
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:642:3: error: 'struct tegra_clk_export_ops' has no member named 'set_rate'
  642 |  .set_rate = gk20a_clk_export_set_rate,
      |   ^~~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:642:14: error: excess elements in struct initializer [-Werror]
  642 |  .set_rate = gk20a_clk_export_set_rate,
      |              ^~~~~~~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:642:14: note: (near initialization for 'gk20a_clk_export_ops')
drivers/gpu/nvgpu/gk20a/clk_gk20a.c: In function 'gk20a_clk_register_export_ops':
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:650:26: error: invalid use of undefined type 'struct tegra_clk_export_ops'
  650 |  if (gk20a_clk_export_ops.data)
      |                          ^
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:653:22: error: invalid use of undefined type 'struct tegra_clk_export_ops'
  653 |  gk20a_clk_export_ops.data = (void *)g;
      |                      ^
drivers/gpu/nvgpu/gk20a/gk20a_sysfs.c: In function 'railgate_delay_store':
drivers/gpu/nvgpu/gk20a/gk20a_sysfs.c:197:3: error: implicit declaration of function 'pm_genpd_set_poweroff_delay' [-Werror=implicit-function-declaration]
  197 |   pm_genpd_set_poweroff_delay(genpd, platform->railgate_delay);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:280: drivers/gpu/nvgpu/gk20a/pmu_gk20a.o] Error 1
make[4]: *** Waiting for unfinished jobs....
drivers/gpu/nvgpu/gk20a/gr_gk20a.c: In function 'gr_gk20a_init_ctxsw_ucode':
drivers/gpu/nvgpu/gk20a/gr_gk20a.c:1931:15: error: 'DMA_ATTR_READ_ONLY' undeclared (first use in this function)
 1931 |  dma_set_attr(DMA_ATTR_READ_ONLY, &attrs);
      |               ^~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/gr_gk20a.c:1931:15: note: each undeclared identifier is reported only once for each function it appears in
At top level:
drivers/gpu/nvgpu/gk20a/gk20a.c:1240:12: error: 'gk20a_pm_resume' defined but not used [-Werror=unused-function]
 1240 | static int gk20a_pm_resume(struct device *dev)
      |            ^~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/gk20a.c:1222:12: error: 'gk20a_pm_suspend' defined but not used [-Werror=unused-function]
 1222 | static int gk20a_pm_suspend(struct device *dev)
      |            ^~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:658:8: error: implicit declaration of function 'tegra_clk_register_export_ops'; did you mean 'gk20a_clk_register_export_ops'? [-Werror=implicit-function-declaration]
  658 |  ret = tegra_clk_register_export_ops(clk_get_parent(c),
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |        gk20a_clk_register_export_ops
drivers/gpu/nvgpu/gk20a/clk_gk20a.c: At top level:
drivers/gpu/nvgpu/gk20a/clk_gk20a.c:638:36: error: storage size of 'gk20a_clk_export_ops' isn't known
  638 | static struct tegra_clk_export_ops gk20a_clk_export_ops = {
      |                                    ^~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:280: drivers/gpu/nvgpu/gk20a/clk_gk20a.o] Error 1
drivers/gpu/nvgpu/gk20a/gr_gk20a.c: In function 'gr_gk20a_add_zbc':
drivers/gpu/nvgpu/gk20a/gr_gk20a.c:3598:3: error: implicit declaration of function 'speculation_barrier' [-Werror=implicit-function-declaration]
 3598 |   speculation_barrier();
      |   ^~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/gk20a_scale.c: In function 'gk20a_scale_make_freq_table':
drivers/gpu/nvgpu/gk20a/gk20a_scale.c:109:8: error: implicit declaration of function 'tegra_dvfs_get_freqs' [-Werror=implicit-function-declaration]
  109 |  err = tegra_dvfs_get_freqs(clk_get_parent(g->clk.tegra_clk),
      |        ^~~~~~~~~~~~~~~~~~~~
drivers/gpu/nvgpu/gk20a/gk20a_scale.c:114:38: error: assignment to 'unsigned int *' from incompatible pointer type 'long unsigned int *' [-Werror=incompatible-pointer-types]
  114 |  profile->devfreq_profile.freq_table = (unsigned long *)freqs;
      |                                      ^
drivers/gpu/nvgpu/gk20a/gk20a_scale.c: In function 'gk20a_scale_notify':
drivers/gpu/nvgpu/gk20a/gk20a_scale.c:228:19: error: 'struct devfreq_dev_status' has no member named 'busy'
  228 |  profile->dev_stat.busy = busy;
      |                   ^
drivers/gpu/nvgpu/gk20a/gk20a_scale.c: In function 'gk20a_scale_init':
drivers/gpu/nvgpu/gk20a/gk20a_scale.c:289:19: error: 'struct devfreq_dev_status' has no member named 'busy'
  289 |  profile->dev_stat.busy = false;
      |                   ^
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:280: drivers/gpu/nvgpu/gk20a/gk20a_scale.o] Error 1
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:280: drivers/gpu/nvgpu/gk20a/gk20a_sysfs.o] Error 1
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:280: drivers/gpu/nvgpu/gk20a/gr_gk20a.o] Error 1
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:280: drivers/gpu/nvgpu/gk20a/gk20a.o] Error 1
make[3]: *** [scripts/Makefile.build:487: drivers/gpu/nvgpu/gk20a] Error 2
make[2]: *** [scripts/Makefile.build:487: drivers/gpu] Error 2
make[1]: *** [Makefile:1011: drivers] Error 2
make: *** [Makefile:515: __build_one_by_one] Error 2

I wanted to do kernel 4.14 LTS but there were more changes that appeared to be less inviting than the 4.4 Civil Infrastructure Platform kernel.

Hope you all have a fantastic night!

I definitely can’t help with anything specific, but I will add some information that might be useful (or frustrating)…

  • Back then NVIDIA had no release for open source of anything related to the driver. I’m not sure what all is open source now, but it is definitely not compatible with the older GPU. The driver is compiled with a give ABI, and the X server dynamically loads this. For the load to succeed, the server has to use that ABI. Moving to a newer server will imply a newer ABI, and so it would fail to use a newer server without a newer driver (which cannot happen).
  • The ChromeBook, although it shares a lot with the Jetson TK1, also has a lot which is different. I would not expect the Jetson TK1 content to work (as a whole) on any ChromeBook. It might even have a variant of the SoC and not the same exact SoC. Definitely the carrier board and device tree and firmware will differ by quite a bit. It isn’t just the Tegra K1 GPU which would need a lot of porting.
  • The changes needed in the kernel would be so pervasive that I doubt porting is practical even if you have significant kernel experience. It isn’t just one feature, but as soon as you find one difference, it’ll turn out that this leads to needing to change another dozen details in the code; and each of those dozen will end up needing a dozen more, and so on (one case where a fractal isn’t so cute). 😔
  • Normally, for a Jetson, it takes a purely Ubuntu rootfs as a start. It then adds NVIDIA drivers and configuration. That step for adding content is what apply_binaries.sh does. That step replaces or edits content in the rootfs. This is in fact the part that makes the sample rootfs compatible with a Jetson, and is also likely something which differs compared to what the ChromeBook uses. I’ve never tried this before, and I don’t know if it will work, but the older apply_binaries.sh might be able to add content to an empty “rootfs/” and you could see what is done. If that fails, then perhaps you could unpack the sample rootfs there, and then delete all files while leaving directories intact, and trying again to apply_binaries.sh. Maybe you’d get to see what is added that way. Regardless of what you find, I suspect that what is there is incompatible with a ChromeBook.
  • All of those gk20a log lines are from GPU driver support. All of that content in the older driver is closed source, and bound not only to the release version, but also to the Xorg ABI.
  • The tegra_dvfs is the digital voltage and frequency scaling…basically control of thermal and power mixes for either saving energy or computing at max load without frying anything from too much heat. This is specific to both the SoC and the surrounding carrier.
  • NVIDIA has a habit of naming the SoC with a major version, and then tacking on a number for a variant. For example, I have an old Tegra 3 here. The SoC is the Tegra 30 because it is the 3 series, and SoC 0. The TK1 for a Jetson is the Tegra 124; this is basically a Tegra 12, and the variant number 4. The Jetson uses a Tegra 124 SoC. I don’t know if the ChromeBook uses that or a different SoC (for example, it would still be a Tegra 12 series, but it might not be the number 4 variant).

I hate to see it stop since you’ve come so far along. I just think you won’t be able to get enough information.

Apologies if I’ve been a bit confusing with my wording and if I’m misunderstanding/mistaking something.

All of the log lines I’ve shown is what has came of a result of me trying to bring over the kernel module to a newer kernel, it has nothing to do with Xorg. Those are compile errors when building a newer kernel with those kernel drivers brought over As far as Xorg is concerned, after a doing a bit of research, perhaps the Xorg legacy package that modern Ubuntu releases also provide. It’s a wrapper for older, legacy GPU drivers, and I’ve yet to see anyone try that, so I might give that a try as well. ABI mismatches don’t always mean Xorg segfaults and breakage to my understanding.

So far, from my testing, it seems like I can actually get the binaries to not destroy the rootfs if I apply them early on, just as you would with the TK1 basic rootfs + binary deployment. I originally thought that the L4T rootfs had Xorg setup and everything, but I stand corrected. It appears that the originally intended route is to apply the binaries to the basic rootfs and then install the desktop, applications, etc. So far, that has been true in my case All of the L4T user space content for the Jetson TK1 appears to work just fine on the Tegra K1 ChromeBook. I should clarify that rootfs that I’ve been deploying to the ChromeBook were cleanly made using debootstrap, nothing specific to any hardware, just a standard Ubuntu/Debian rootfs creation tool that works for any supported architecture (x86, armhf, arm64, etc). I’ve taken a look at how people usually deploy Ubuntu on the Jetson TK1, and I think what has happened is that breakage happens once you make too many changes prior to deploying the binaries (such as installing Xorg packages). I’ve noticed that several glibc configuration changes occur once you start adding things to the system, but no breakage occurs if you have applied the binaries prior to changing/adding anything else to the system; perhaps this relates to the post I saw where someone’s rootfs kept corrupting with their TK1 after using the binary deployment script twice on the same rootfs. I should also clarify that the working Ubuntu Mate Jammy installation on said ChromeBook is using the L4T binaries (including the GPU blobs), designed for the Jetson TK1, deployed using the same script just as one would on a TK1, something like: 'sudo ./apply_binaries.sh -r [ROOTFS LOCATION HERE]"

The reason why I figured that only the GPU driver would need porting is that I read that the the ChromeBook has decent enough support in mainline Linux both in userspace and kernel space, it’s just that the Nouveau kernel driver most probably won’t work with the blobs (despite being at least somewhat based on the open source L4T GPU kernel driver) and that the Nouveau user space in MESA is severely broken in upstream, in fact, it seems like the code in MESA has been bitrotting for years and no one has bothered to fix it. Anything that works on that device should work on the Jetson TK1. According to several people and the Debian Wiki, I can determine that mainline is also pretty good for the Jetson TK1 (just don’t expect working Nouveau as again; it’s broken in MESA). I took a look at Torvald’s Linux Git repo and just as I had expect, much of NVIDIA’s open source GK20A code is used in the Nouveau kernel driver, but it hasn’t appeared to have been updated in several years (though that shouldn’t necessarily matter). I should also note that the only device-specific things that I have been using on said ChromeBook were the ChromeOS kernel (and looking at the GPU driver code between the TK1 kernel source and the ChromeOS kernel, it doesn’t appear that anything device specific was enacted whatsoever, which is expected). I’m pretty sure the Jetson TK1 kernel would work on it as well if I really wanted to. In fact, looking at the TK1 kernel source, there’s a “chromeos” folder just like in Chrome OS kernel. Google based that Chrome OS kernel on NVIDIA’s kernel from what I can tell. Though, the last time I tried porting the GPU driver over for the GK20A was for that 4.4 CIP (Civil Infrastructure Platform) kernel, which will only be good as a starting base when you look at it realistically. Getting it to 6.1 LTS will take a lot of work most certainly, though I would most certainly like to see that happen.

The only difference between the SoC in the ChromeBook and Jetson TK1 I saw (according to someone’s notes) last I remember are none. Said notes were on a GitHub project surrounding deploying modern Ubuntu and Debian on ChromeBooks, primarily ARM ChromeBooks. However, the device tree file in the Linux kernel (which can be very much easily changed) has the CPU clocked lower for the ChromeBook, but that’s likely because it’s a laptop.

At the very least: It seems like people were able to get newer Ubuntu running on the Jetson TK1 with the NVIDIA 3.10 L4T kernel, but they were getting stuck with Xorg related errors; errors that I had encountered as well. I’m going to try the Xorg legacy package and see if that wrapper gets modern Xorg running with the older GPU (even if there’s an ABI mismatch), but whether be using the Xorg wrapper (if it does work) or by using downgraded Xorg packages, there shouldn’t be a reason why people can’t get modern Ubuntu running on their Jetson TK1’s with working GPU acceleration via the NVIDIA blobs

Also, update on the Bionic Xorg testing, bypassing the ABI mismatch prevention does not fix it. The screen freexes with a line cursor on the top left that is static. It looks like there was a giant change in Xorg. These days, ABI mismatches aren’t a big deal (e.g. Mid 2010 MacBook Pro with an NVIDIA GeForce 320M + Arch Linux + NVIDIA 340 legacy DKMS drivers, patched to work on newer kernels + Xorg with ABI mismatch prevention disabled; all working well). That could probably be attributed to the fact that Xorg hasn’t seen massive development in quite awhile. Xenial Xorg is the last Xorg to work with the NVIDIA proprietary blobs, which thankfully, does work with Ubuntu Jammy, and for that matter, probably on Ubuntu Mantic as well. There’s a few debs that I need to patch to prevent some more dependency conflicts for certain use cases; at least I got the Ubuntu Mate desktop running. Had to use squashfuse to manually open the Chromium snap package, so I could then use Chromium. I’ll note, Compiz being turned off makes applications way more faster, including Chromium. If you do use Compiz on newer builds of Ubuntu with this unique kind of setup (with the downgraded Xorg and all) will you need to enable triple buffering in the Xorg conf to prevent artifacting (affects anything that uses Compiz, not just Mate but also the Unity desktop).

Also, I found the official developer information for Chrome OS devices which confirms it indeed uses a Tegra 124.

Developer Information for Chrome OS Devices

I was just adding notes in general, they were not about the specific issues.

If a kernel module is not compiled against a kernel with the same integrated features (those with “=y”, compared to not installed of modules with “=m”), then it is unlikely to correctly load. However, let’s take the case of some of the errors.

Those in this format:

error: "CONFIG_DEBUG_FS" is not defined

…are likely because you did not enable that feature in the recipient kernel’s config. If you take that kernel, and use a configuration editor (which knows about dependencies), you could try enabling this. Editors such as the nconfig build target have a symbol search so you could search for “debug_fs”.

Errors with “tegra” or “gk20amight be from a configurable missing function, and it is possible that enabling the correct symbol would provide that function. I don’t know which symbol, but I can tell you that some of that content is available in the mainline kernel, but most of that is out of tree content. You will find a lot of _ddot_ macros in configurations which are just a file path “../” referring to the parent directory. Much of the tegra and NVIDIA content configuration descends below the kernel’s top directory, and then reascends into the NVIDIA out of tree content. The 5.x kernels used in 20.04 don’t have that content in mainline, but it does exist in the L4T R35.x source. For downloads by L4T release, e.g., L4T R35.4.1 (which has out of tree content and a 5.x kernel source), check:
https://developer.nvidia.com/embedded/jetson-linux-archive

The part which gets more difficult: The internal functions being called from any given kernel feature might change signature between major releases. Other functions might have been added, and some might no longer exist. You could get lucky, and all of those can be added by knowing the correct configuration name (the “symbol”, the “CONFIG_...something...”). However, you’re looking at ARMv7-a (32-bit ARM), and although something else may have triggered adding that version of whatever is missing, NVIDIA only added ARMv8-a (64-bit ARM). Granted, many functions don’t care if it is ARMv7-a or ARMv8-a, but I suspect some of the NVIDIA out-of-tree content will. About the only way to know is to find the config which adds the missing functions (the CONFIG_DEBUG_FS is easy since it tells you exactly which symbol, but that is not a function, it is a name for a group of functions).

If you take a look at something like this:

error: implicit declaration of function 'nvhost_module_disable_clk'; did you mean 'nvhost_module_idle_ext'? [-Werror=implicit-function-declaration]
 1513 |  nvhost_module_disable_clk(&dev->dev);

…you’ll have to experiment. Or figure out a symbol which provides nvhost_module_disable_clk. Maybe that no longer exists in the 5.x kernels. Maybe there is an internal function like 'tegra_dvfs_get_freqs' which does not exist.

This error:

error: 'struct tegra_clk_export_ops' has no member named 'enable'

…is a bit more ominous. It looks like the function in the preexisting module depends on a changed struct. You could add that member, but the outside world in the rest of the kernel won’t use it if you add it. You’d have to find every location which uses that struct, and figure out what the consequences are, and maybe edit those functions. Some of those might be closed source.

You’ve done a lot for porting, but the closed source ARMv7-a 32-bit, combined with the 5.x kernel only being updated for 64-bit, means you might need to use some extraordinarily imaginative function edits. At least if you want to use the GPU. There might be ways to use Mesa for software rendering. Those people I’ve heard of getting the newer releases running did not have GPU available for hardware rendering, nor for CUDA.

The whole point about the driver which plugs into Xorg is this: That Xorg is not just for rendering. There are AI applications which sometimes used the Xorg server as an API and not for rendering. People mistake the X server for being a graphics server, but it is really more than that; that server is an ABI in user space for GPU access, and it doesn’t particularly care if there is a monitor attached to the rendering buffer, nor if the content is graphics. It is a defined API to talk to the GPU (or a framebuffer for software rendering). Not having this implies there are some CUDA apps which won’t work with GPU. If you care about CUDA, then you have to care about the X server.

Thanks for the info and insights, linuxdev.

I have zero experience with working with Xorg’s at such an in-depth level (such as working with it’s code and all that). I’m sure that getting the blobs to run on a newer Xorg is most probably possible with a ton of time, experience, bisecting, and all of that effort to patch Xorg extensively, but it would take lot of work, probably more than what it’s worth considering how Xorg is going to be on its way out in a couple of years. The best I can do surrounding Xorg is use Xenial Xorg’s packages with the blobs, on Ubuntu Jammy, and well, at least for now, with that old kernel, and it does work correctly. I’ve tried a few OpenGL applications and GPU acceleration does work doing this.

I should go out and properly test CUDA though. I have yet to install JetPack. I’m sure that it’ll work though. It also seems like that Xorg 1.18 and 1.19 are massively different ABI-wise to the point of outright total incompatibility, hence why Bionic’s Xorg will not work with the proprietary blobs but Xenial’s does. Xenocara is an Xorg fork based on Xorg 7.7, I wonder if that could help or be a solution that could succeed the use-Xenial-Xorg-packages-on-Ubuntu-Jammy solution. I looked at the GitLab for MESA and the issues page. It seems like Nouveau never really worked for long if at all without regressions on the GPU. Getting Nouveau to work somewhat with artifacts requires a downgraded MESA and Xorg as it’s that severe. Maybe someday someone will fix that (hopefully) if there’s enough interest to fix it, or maybe it’ll just continue down its path of bitrot, who knows?

I also figured out a factor in why ./apply_binaries.sh was messing up with my rootfs. Starting with some newer version of Ubuntu, there’s some symlink business that happens between /usr/lib and /lib that wasn’t present before and it looks like the script gets confused with that and does an unlink which makes chrooting into the system impossible without fixing it. Starting with an older Ubuntu rootfs that doesn’t rely on this kind of symblink works just fine and doesn’t have this issue.

Surrounding the kernel driver porting, considering how the ‘nvgpu’ driver is open source, I thought someone would’ve already tried to port it over. Similar has been done for HardKernel’s Odroid XU3/XU4 users. The XU3 was released all the way back in 2014. There were efforts to port the ancient Mali kernel space over to newer kernels and such efforts were proven to be successful (all the way up to Linux 6.1 LTS). Surrounding the ‘nvgpu’ driver, there was some out-of-tree patches that was done to the kernel that weren’t so obvious that got it to build a little further. I noticed the recurring theme of out-of-tree power management code and power management code that has just been simply absent as such was deprecated in favor of something else.

Interestingly enough, I believe I have found a release of L4T 24.04 for the Tegra K1 (124) in the ChromeOS installation on the ChromeBook. ChromeOS doesn’t rely on Xorg, it relies on Wayland. After some non-extensive ‘testing’ does it appear that the NVIDIA ‘libdrm’ from ChromeOS contains a number of functions you need for GBM. Not sure if this could be useful for those who want to try running Wayland environments. Assuming you could, likely chance is that the libraries would be too old to run anything without injector hooks (which is the case for old Mali Midgard Wayland blobs + modern Wayland)–xwayland would most definitely not work or work well assuming I could get those NVIDIA binaries to run at all.