I don’t know how much experience you have with Linux itself, but it comes in handy. So forgive me if I mention things you already know.
64-bit/32-bit
There are actually two environments to consider if programming entire operating systems. One is kernel space, the other is user space. Most of the time the two are both the same instruction set, so both are either 64-bit or 32-bit…most of the time.
A desktop x86_64 CPU is x86_64 architecture, and 64-bit. On the desktop there is a compatibility mode for 32-bit, usually called i686 (i386 up to i686 were of course all 32-bit, but the newer i686 supported additional features). When the Intel format CPUs first released 64-bit versions they were concerned with backwards compatibility with 32-bit PCs. Basically old 32-bit compatible hardware was glued on to the 64-bit hardware. When you see software available for a desktop PC from most package management flavors you will typically see 64-bit versions of the package.
The 64-bit architecture is the native architecture for the PC. 32-bit is only valid if you’ve supplied “compatibility” libraries and linkers for user space. If you’ve added 32-bit compatibility, then the package manager has been told about a foreign architecture you are interested in…this is despite the fact that 32-bit is built in to the CPU. The CPU goes into a compatibility mode in order to service 32-bit (it isn’t a software adapter, it is a hardware adapter).
The TX1 is 64-bit ARMv8-a, but has a 32-bit compatibility mode similar to the desktop PC CPU…“ARMv8” without the “-a” is 32-bit, “ARMv8-a” is 64-bit. Kernels and kernel space and drivers always run 64-bit, this is native. To run 32-bit the CPU goes into compatibility mode. This is a foreign architecture directly supported by the ARM Cortex A57 cores.
To use 32-bit you have to be in user space. This in turn implies drivers and kernel space still run 64-bit. The linkers and libraries are 32-bit. ARMv8 32-bit is a super set of the older 32-bit ARMv7. All ARMv7 instructions will run on ARMv8, but not all ARMv8 instructions will run on ARMv7…though they might for the bulk of instructions.
32-bit on the first TX1 was used in user space because it was the moment of transition when most previous packages and applications had been built for 32-bit and were not ported to arm64/aarch64/ARMv8-a packages. Drivers and kernel space were still 64-bit, and the hardware is still 64-bit.
Very soon after arm64/aarch64 appeared on the market packages began arriving which were built for this architecture. NVIDIA released an upgrade in which both user space and kernel space were 64-bit. No 32-bit compatibility mode is added, although you could add 32-bit libraries and linkers in user space and tell your package manager you are interested in seeing that foreign architecture (armhf/arm32). This would probably be a bad choice on an embedded system, especially since storage is limited and since the 32-bit compatibility mode is quite slow.
The next release, being fully 64-bit in both user space and kernel space was a big improvement since there were actually 64-bit packages available. This is with a 3.x version kernel. The R28.1 release has not only pure 64-bit, but it has also migrated to the newer generation 4.x kernel.
Note that adding a foreign architecture to a native architecture is a “Linux thing”, and not specific to Jetsons. On a desktop PC this is simple because the compilers have integrated 64-bit and 32-bit support in a single compiler (linkers and libraries still must be a separate addition). There is very little difference between adding i686 support on a PC and adding cross-compile armhf support on a PC…only the desktop PC CPU can’t switch to armhf and execute it. User space support for cross compile of ARMv8-a gets a bit trickier because compatibility mode causes the need to support not just 64-bit tools, but also 32-bit tools…and compilers and tools are not integrated for these architectures the way they are integrated on a PC.
Even if you ignore 32-bit/64-bit compatibility and features I would always recommend flashing before you try to do any real work because the initial releases were never all that great, and the latest 64-bit release is quite good.
Flash Software
So far as flash software goes JetPack is a front end to the flash script. The flash script only requires any x86_64 Linux and is not dependent upon Ubuntu (I use Fedora, I don’t like Ubuntu for development, and Ubuntu cannot be installed on my desktop due to bugs in its installer regarding BIOS style partitions). The Ubuntu 14.04 requirement is just for JetPack. Part of the reason for this is that JetPack can manage packages for cross-platform work on both host and the Jetson. It’s hard to create a general utility for a platform if you don’t know which platform it is…creating packages for simultaneous use of Fedora+SuSE+CentOS+Debian+Ubuntu+WhateverGreatFlavor would not be easy.
Ubuntu 14.04 is supported for use as a host, and though not supported, many people have reported that Ubuntu 16.04 works fine for the most part…most issues are related to packages which won’t work on 16.04 hosts. Unfortunately there are some software packages which at first seem available only via JetPack. I say “seem” because on any x86_64 host you can extract the URLs of those packages and download them with wget…manual install order issues can be a pain though, whereas JetPack handles install order issues for you.
Word of caution on flash software on a VM: Many people use a VM and discover a VM typically does not have well-behaved USB (technical term: “it sucks”). VMs are not supported, but the people who have made this work typically have to fine-tune USB, e.g., specifically mark the Jetson in recovery mode for pass-through, specifically mark to run in USB2 mode, and add additional buffer to the USB port.
Should you choose to flash on command line without JetPack you would need the driver package plus sample rootfs. Both the TX1 and TX2 use the same sample rootfs in this release, but the driver package is different. Be sure on driver package to pay attention to TX1 or TX2 driver package version if you go this route. The R28.1 URL is here (you may need to log in separately, not sure):
https://developer.nvidia.com/embedded/linux-tegra
Flashing
For both JetPack and command line make sure your Linux host uses a native Linux file system type. NTFS/VFAT/others do not work since they cannot preserve Linux permissions. I think Live DVD distributions also would require a separate mounted Linux file system type, e.g., ext4.
For JetPack you just unpack as a regular user and run it as a regular user…you’ll be prompted at some point for a password to use sudo.
If you do flash on command line you’ll see images are indeed used. Typically it would go like something like this:
- Make sure you have lots of space.
- Unpack the driver package as a regular user.
- Make sure you still have about 20GB or more of space.
- cd to "Linux_for_Tegra/rootfs/" and use sudo to unpack the sample rootfs as user root.
- cd back one directory up to the "Linux_for_Tegra/" directory.
- Unpack the sample rootfs with sudo (as user root).
- Run "sudo ./apply_binaries.sh"
- Put the Jetson in recovery mode, attach the supplied micro-B USB cable.
- Make sure the TX1 is visible to the host, run "lsusb -d -0955:7721" and see if something shows up...it should.
- Run this command and wait...it takes a long time:
<b>sudo</b> ./flash.sh -S 14580MiB jetson-tx1 mmcblk0p1
During the flash the rootfs folder is used for the basis of creating an image for the root partition (“APP” partition in config files). Depending on other arguments part of the boot environement is adjusted, e.g., some files in “/boot” will change versus the rootfs “/boot” directory, but everything else will be verbatim.
apply_binaries.sh overlays the NVIDIA hardware drivers onto the rootfs folder. Without this the sample rootfs is pure Ubuntu. Once you’ve added the NVIDIA drivers it can be referred to as L4T (Linux for Tegra). Even so, it is still Ubuntu and you can use any Ubuntu admin book/information directly on the Jetson.
The flash.sh script has an option “-r” to reuse the image. In this case it takes less time because you don’t have to generate a huge image. It also allows to restore a system from a clone of the rootfs image if you have previously backed up via cloning.
You will see subdirectory “bootloader/”. Before a flash you will not see any “system.img*” file. After a flash you will see “system.img.raw” and “system.img”. “system.img.raw” is the entire file system bit-for-bit and can be loopback mounted, edited, so on. “system.img” is a “sparse” (basically compressed) version of this. flash.sh option “-r” would imply reusing the existing system.img file. What comes in handy is that if you place an uncompressed system.img.raw there, and name it system.img, then flash works perfectly fine even with the uncompressed version (just make sure the name is system.img). system.img and system.img.raw take up a lot of space on the host, but if you are going to flash without reusing the image, then you can consider these temporary files and delete them when done. If you are reusing those files then you don’t need the content of the “rootfs/” subdirectory anymore…you could delete those files and unpack sample rootfs again at a later date if you want to generate a new rootfs image.
Do note that flash uses only the micro-B USB cable. JetPack is a front end to flash and package management, so during flash stage no ethernet cable is required. After flash, when package management begins, ethernet is required, and it must be wired ethernet, not WiFi.
Miscellaneous
A really big difference between a Jetson and most embedded systems is that you are creating an entire Linux system, not a bare bones system. The sample rootfs is not a cut down customized system, it is an ordinary Ubuntu system. This tends to mean you will need to know how to be an administrator (to some small extent) on Ubuntu.
Probably your questions about missing packages would be best studied by knowing “apt” and “apt-get” are the normal Ubuntu package tools. The file “/etc/apt/sources.list” contains a list of standard Ubuntu repositories used for package search…for many packages you will have to uncomment some of the optional repositories.
After changing sources.list (or perhaps before any update) you should run “sudo apt update”. This creates a snapshot at an instant in time for what is visible.
If you wanted to search for a package, e.g, git, you’d run:
apt search git
If you wanted to install git:
sudo apt-get install git
To be fair, this is the only embedded system which is basically a true and full operating system capable of competing with a desktop in such a broad range of uses…but it is still an embedded system. Space is never free without a trade-off. A desktop system can afford to install every kernel module there is for your acme elephant elevator controller and wyle e. coyote roller blade controller…or for the AMD graphics card and the Intel 10G-base-T driver. It would be a terrible choice if your Jetson arrived with all of that on it the way a desktop does. Very few people develop natively on their embedded product when it is something less powerful. So consider that despite the resemblance to a desktop in function that you won’t get as many pre-installed packages just because it is an embedded product.
If you want to verify the NVIDIA drivers are correctly in place (they don’t use apt or package management):
sha1sum -c /etc/nv_tegra_release
Note that on rare occasions an Ubuntu package could overwrite “/usr/lib/xorg/modules/extensions/libglx.so” (if video breaks after an update this would be the cause). You can restore this by deleting whatever version of libglx.so was put there, and then adding a symbolic link to “/usr/lib/aarch64-linux-gnu/tegra/libglx.so” (this is the hardware accelerated version). Example:
cd /usr/lib/xorg/modules/extensions
sudo ln -sf /usr/lib/aarch64-linux-gnu/tegra/libglx.so <b>.</b>
I would definitely recommend keeping packages up to date, don’t avoid updates. Also, if your Jetson is visible on a network that isn’t yours beware that the passwords are defaults known to the entire world. “ubuntu/ubuntu” and “nvidia/nvidia” name/pass should be changed before touching the outside world.