Creating RAID 10 with 4 USB devices

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
21

So, I managed to get the script(s) done by December (next week is Turkey week), and so I have posted it to github and pypi. It automates the whole process System image creation and only requires SDK Manager and your board appropriate bundle be installed. So far it has only been tested on the Jetson Nano Development board. Later I will add support for Xavier (and possibly the rest of Nvidia’s boards).

You need only run sudo tegrity -v --menuconfig for your purposes after installation, and it should spit out a flashable sd card image when done. The ‘-v’ is for verbose and completely optional. The installation process will configure all other system requirements for you (toolchain, apt requirements, kernel source, etc…). There are other options which are described in the documentation, but they lack testing. --rootfs download seems to work fine if you want to reset the rootfs to the stock one Nvidia provides. This is very alpha software at the moment, so please report any issues you find.

Next feature being worked on is first boot script instalation and using that to install debian packages on first boot.

22

Hey, I am interested if anything comes of this.

Is it possible to use all 4 USB3.0 ports at once with maximum speed? I am hoping to be able to put 4 external SSDs together to accelerate Machine Learning and AI capabilities of this board.

23

Hi mdegans

I have to install SDK-Manager and your scripts on Jetson Nano?
I think no, I have to install these things to another Ubuntu host, right?

But I get this ERROR with “sudo ./install.py”:

ERROR:tegrity.toolchain:aarch64-linux-gnu toolchain not found. Do you wish to install one, either from system apt repositories or the recommended from releases.linaro.org?
Traceback (most recent call last):
  File "./install.py", line 56, in <module>
    cli_main()
  File "./install.py", line 52, in cli_main
    install(args.prefix)
  File "./install.py", line 14, in install
    tegrity.utils.ensure_system_requirements()
  File "/home/marco/JETSON/tegrity-master/tegrity/utils.py", line 233, in ensure_system_requirements
    return tegrity.toolchain.ensure(cross_prefix)
  File "/home/marco/JETSON/tegrity-master/tegrity/toolchain.py", line 155, in ensure
    while not 1 <= choice <= 2:
TypeError: '<=' not supported between instances of 'int' and 'NoneType'

Thank you for your help.

24

Thanks for reporting that. I see what’s wrong will fix it at the next release.

A workaround to get the installer working should be

sudo apt install gcc-aarch64-linux-gnu
sudo ./install.py
sudo tegrity-toolchain --install-tarball

The first command installs the default aarch64 toolchain from Ubuntu, just to bypass this check.
Second installs my scripts and python package.
Third installs Nvidia’s recommend toolchain in /use/local/bin

25

SDK Manager and my software must be installed on the Ubuntu host for the moment, but I may remove SDK Manager as a dependency at some point so it can work on Tegra platforms as well.

Windows Pro and Docker support is also planned, but no firm date on that yet.

26

Now, I get another ERROR:

# sudo ./install.py

...
...
...

Processing triggers for libc-bin (2.27-3ubuntu1) ...
INFO:tegrity.utils:Ensuring /home/marco/.tegrity exists
Traceback (most recent call last):
  File "./install.py", line 56, in <module>
    cli_main()
  File "./install.py", line 52, in cli_main
    install(args.prefix)
  File "./install.py", line 20, in install
    tegrity.utils.run(command).check_returncode()
  File "/home/marco/JETSON/tegrity-master/tegrity/firstboot.py", line 167, in _run_one
    return subprocess.run(*args, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 423, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/pip3': '/usr/bin/pip3'

# sudo tegrity-toolchain --install-tarball
sudo: tegrity-toolchain: command not found

What I’m doing wrong?

27

Nothing. Thanks for testing. It’s alpha software with few tests written (yet). The problem is Pip3 needs to be installed. I will add it to the dependencies check.

You can fix the error with “sudo apt install python3-pip”

28

Ok, I was able to install it successfully now:

...
...
...
Installing collected packages: tegrity
  Running setup.py install for tegrity ... done
Successfully installed tegrity-0.0.3

But something doesn’t work with tegrity:

# sudo tegrity -v --menuconfig
usage: tegrity [-h] [--cross-prefix CROSS_PREFIX]
               [--firstboot FIRSTBOOT [FIRSTBOOT ...]]
               [--public-sources PUBLIC_SOURCES]
               [--public-sources-sha512 PUBLIC_SOURCES_SHA512]
               [--build-kernel] [--kernel-load-config KERNEL_LOAD_CONFIG]
               [--kernel-save-config KERNEL_SAVE_CONFIG]
               [--kernel-localversion KERNEL_LOCALVERSION]
               [--kernel-menuconfig] [--rootfs-source ROOTFS_SOURCE]
               [--rootfs-source-sha512 ROOTFS_SOURCE_SHA512] [-o OUT]
               [-l LOG_FILE] [-v]
               l4t_path
tegrity: error: the following arguments are required: l4t_path
29

You need to supply the path to the Linux_for_Tegra folder SDK Manager installs. I used to autodetect this but removed this feature temporarily.

Usually this will be ~/nvidia/nvidia_sdk/$(JETPACK_VERSION)/Linux_for_Tegra

The path is configured when running SDK Manager to install a bundle for your specific board. You can skip the flashing stage when prompted.

30

You mean this “Linux_for_Tegra” folder is generated after SDK-Manager is installed?

I think I did not install the SDK-Manager in a correct way:

marco@ubuntu18:~/JETSON$ sudo apt install /home/marco/JETSON/sdkmanager_1.0.0-5517_amd64.deb 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'sdkmanager' instead of '/home/marco/JETSON/sdkmanager_1.0.0-5517_amd64.deb'
The following NEW packages will be installed:
  sdkmanager
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
After this operation, 280 MB of additional disk space will be used.
Get:1 /home/marco/JETSON/sdkmanager_1.0.0-5517_amd64.deb sdkmanager amd64 1.0.0-5517 [63.7 MB]
Selecting previously unselected package sdkmanager.
(Reading database ... 170489 files and directories currently installed.)
Preparing to unpack .../sdkmanager_1.0.0-5517_amd64.deb ...
Unpacking sdkmanager (1.0.0-5517) ...
Setting up sdkmanager (1.0.0-5517) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for desktop-file-utils (0.23-1ubuntu3.18.04.2) ...
Processing triggers for gnome-menus (3.13.3-11ubuntu1.1) ...
N: Download is performed unsandboxed as root as file '/home/marco/JETSON/sdkmanager_1.0.0-5517_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
marco@ubuntu18:~/JETSON$

Do you know how to proceed?

Many thanks.

31

Yes, the Linux_for_Tegra folder installed after SDK Manager is installed, you run SDK Manager, and download the bundle for Nano development version.

Apt does some things as user _apt and it looks like it can’t read your home. That’s fine. I think it probably worked anyways since I believe it falls back to root.

If you check your app chooser (usually Windows key) and somehow SDK Manager is not installed when you search, then you might try installing it the SDK Manager .deb with “sudo dpkg -i sdkmanager_1.0…”

If it is installed, run SDK Manager and install the software for Jetson Nano development version, choosing to option to skip flash (or skip it when prompted). Then you should have the Linux_for_Tegra folder somewhere in ~/nvidia provided you don’t change the default hidden at the bottom of the SDK Manager Window.

32

Just to emphasize, the “.deb” install package adds a program. Running the program adds things to your “~/nvidia/” directory. The further you go in actually flashing or adding packages with SDKM, the more gets added to your host PC.

33

Yeah, the stale JetPack folders can use up a lot of space if you don’t delete old versions, especially if you have multiple Linux_for_Tegra/rootfs folders (eg. backups).

Anyway, schmuckimarco89, don’t worry if you can’t find the Linux_for_Tegra folder. I am adding a gui to my scripts that creates and validates a configuration for you so, so you won’t have to worry about finding any folders. There is a dropdown and you just select your JetPack. A sneak peek:

1017×636 50.3 KB

It will also run gconfig for you, instead of menuconfig, so you can use a gui to configure the kernel, if you choose to rebuild the kernel. It’ll be at least few weeks, probably, before I’m finished, and tests are written, but, when finished it will hopefully make it easier for people to build and customize their own images using Nvidia’s software under the hood. You’ll be able to add your own software, users to run said software, and add systemd units for your software so it runs on boot.

I’m separating the gui from the underlying software so the gui can run on multiple platforms, like Windows or Mac OS. The idea is that the gui creates a configuration defining an image and that config is then fed into a virtual machine or Docker (which runs in a VM on most platforms) with SDK Manager installed. Then out pops an image you can flash with Etcher and/or distribute.

34

Hi mdegans

Now, I have the folder after running sdkmanager:

ls -la /hw_image/JetPack_4.3_Linux_P3448/Linux_for_Tegra/
total 208
drwxrwx--- 1 root vboxsf   704 Dez 10 08:04 .
drwxrwx--- 1 root vboxsf    96 Jan 25  2020 ..
-rwxrwx--- 1 root vboxsf 15187 Dez 10 07:58 apply_binaries.sh
drwxrwx--- 1 root vboxsf  1216 Dez 10 08:01 bootloader
-rwxrwx--- 1 root vboxsf  3016 Dez 10 07:58 build_l4t_bup.sh
-rwxrwx--- 1 root vboxsf 76773 Dez 10 07:58 flash.sh
drwxrwx--- 1 root vboxsf   416 Dez 10 08:01 kernel
-rwxrwx--- 1 root vboxsf 10328 Dez 10 07:58 l4t_generate_soc_bup.sh
-rwxrwx--- 1 root vboxsf 28546 Dez 10 07:58 nvmassflashgen.sh
drwxrwx--- 1 root vboxsf   832 Dez 10 07:58 nv_tegra
-rwxrwx--- 1 root vboxsf  3098 Dez 10 07:58 p2371-2180-devkit-24x7.conf
-rwxrwx--- 1 root vboxsf  3442 Dez 10 07:58 p2371-2180-devkit.conf
-rwxrwx--- 1 root vboxsf  1674 Dez 10 07:58 p3448-0000.conf
-rwxrwx--- 1 root vboxsf  3637 Dez 10 07:58 p3448-0000.conf.common
-rwxrwx--- 1 root vboxsf  1858 Dez 10 07:58 p3448-0000-emmc.conf
-rwxrwx--- 1 root vboxsf  1693 Dez 10 07:58 p3448-0000-sd.conf
-rwxrwx--- 1 root vboxsf  4429 Dez 10 07:58 README_Massflash.txt
drwxrwx--- 1 root vboxsf    96 Dez 10 07:58 rootfs
drwxrwx--- 1 root vboxsf    96 Dez 10 07:58 source
-rwxrwx--- 1 root vboxsf  9665 Dez 10 07:58 source_sync.sh
drwxrwx--- 1 root vboxsf   224 Dez 10 08:01 tools
-rwxrwx--- 1 root vboxsf 24963 Dez 10 07:58 TX1_boot-firmware-redundancy.txt

But I got some errors during running of sdk-manager:

16:04:28 ERROR : Drivers for Jetson TX1/Nano : tar:
16:04:28 ERROR : Drivers for Jetson TX1/Nano : Linux_for_Tegra/jetson-nano-qspi.conf: Cannot create symlink to ‘p3448-0000.conf’
16:04:28 ERROR : Drivers for Jetson TX1/Nano : : Read-only file system
16:04:28 ERROR : Drivers for Jetson TX1/Nano :
16:04:28 ERROR : Drivers for Jetson TX1/Nano : tar:
16:04:28 ERROR : Drivers for Jetson TX1/Nano : Linux_for_Tegra/jetson-nano-qspi-sd.conf: Cannot create symlink to ‘p3448-0000-sd.conf’
16:04:28 ERROR : Drivers for Jetson TX1/Nano : : Read-only file system
16:04:28 ERROR : Drivers for Jetson TX1/Nano :

16:05:19 ERROR : Drivers for Jetson TX1/Nano : tar:
16:05:19 ERROR : Drivers for Jetson TX1/Nano : Linux_for_Tegra/jetson-tx1.conf: Cannot create symlink to ‘p2371-2180-devkit.conf’
16:05:19 ERROR : Drivers for Jetson TX1/Nano : : Read-only file system
16:05:19 ERROR : Drivers for Jetson TX1/Nano :
16:05:19 ERROR : Drivers for Jetson TX1/Nano : tar:
16:05:19 ERROR : Drivers for Jetson TX1/Nano : Linux_for_Tegra/jetson-nano-emmc.conf: Cannot create symlink to ‘p3448-0000-emmc.conf’
16:05:19 ERROR : Drivers for Jetson TX1/Nano : : Read-only file system
16:05:19 ERROR : Drivers for Jetson TX1/Nano :
16:05:19 INFO : TensorRT on Target : verifying checksum of /media/sf_ubuntu_shared_folder/sdk_download/libnvonnxparsers6_6.0.1-1+cuda10.0_arm64.deb
16:05:20 ERROR : Drivers for Jetson TX1/Nano : tar: Linux_for_Tegra/bootloader/tos.img: Cannot create symlink to ‘tos-mon-only.img’: Read-only file system

16:06:01 ERROR : Drivers for Jetson TX1/Nano : tar:
16:06:01 ERROR : Drivers for Jetson TX1/Nano : Exiting with failure status due to previous errors
16:06:01 ERROR : Drivers for Jetson TX1/Nano :
16:06:01 INFO : Drivers for Jetson TX1/Nano : exit status 2
16:06:01 INFO : Drivers for Jetson TX1/Nano : [ Package Install Finished with Error ]
16:06:01 INFO : Drivers for Jetson TX1/Nano : [ 540.00 KB used. Disk Avail: 38.62 GB ]
16:06:01 INFO : Drivers for Jetson TX1/Nano : [ NV_L4T_DRIVERS_T210_COMP Install took 1m56s ]
16:06:01 ERROR : Drivers for Jetson TX1/Nano : Run commands failed at step Install: command /tmp/tmp_NV_L4T_DRIVERS_T210_COMP.sh finished with error
16:06:01 INFO : Drivers for Jetson TX1/Nano :
16:06:01 ERROR : Drivers for Jetson TX1/Nano : command terminated with error
16:06:01 ERROR : Drivers for Jetson TX1/Nano : install 'Drivers for Jetson TX1/Nano' failure, command < cd '/media/sf_ubuntu_shared_folder/hw_image/JetPack_4.3_Linux_P3448' 

16:23:09 ERROR : Unavailable components detected, please check log
16:23:09 ERROR : File System and OS : File System and OS Skipped. Due to its dependencies fail to install: Drivers for Jetson TX1/Nano,
16:23:10 ERROR : Device Mode Host Setup in Flash : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : Flash Jetson Nano : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : Device Mode Host Setup in Target SDK : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : DateTime Target Setup : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : CUDA Toolkit for L4T : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : cuDNN on Target : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : TensorRT on Target : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : OpenCV on Target : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : VisionWorks on Target : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : VPI : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : NVIDIA Container Runtime with Docker integration (Beta) : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : Multimedia API : Flash skipped due to component(s) download/install failure
16:23:10 ERROR : DeepStream : Flash skipped due to component(s) download/install failure

Should I continue with the below command?

sudo tegrity -v --menuconfig /hw_image/JetPack_4.3_Linux_P3448/Linux_for_Tegra/

Many thanks for your help!

35

You installed with SDK Manager? As your own user? If so, the permissions are wrong. It looks like you’re trying to install to a vbox share folder, which can’t work.

If you run the scripts now, they probably won’t work because some important symlinks couldn’t be created and the rootfs in particular needs to be owned by root:root.

You might try choosing another path, within your home. You can skip the flashing part, but the symlink errors are likely fatal to the installation.

36

Hi mdegans

You’re right. If I’m not changing the directories in SDK-Manager, it’s working.

Following a summary what I did:

A. Install SDK-Manager:

# sudo apt install /path/sdkmanager_1.0.0-5517_amd64.deb

B. Run SDK-Manager:

# sdkmanager

(download and install nvidia stuff, skip flashing)

C. Install tegrity sw:

# sudo apt install python3-pip
# sudo apt install gcc-aarch64-linux-gnu
# sudo ./install.py

D. Build Nvidia Jetson Nano Image:

# sudo tegrity -v --menuconfig /home/marco/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra

But something doesn’t work at point D:

# sudo tegrity -v --menuconfig /home/marco/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra
usage: tegrity [-h] [--cross-prefix CROSS_PREFIX]
               [--firstboot FIRSTBOOT [FIRSTBOOT ...]]
               [--public-sources PUBLIC_SOURCES]
               [--public-sources-sha512 PUBLIC_SOURCES_SHA512]
               [--build-kernel] [--kernel-load-config KERNEL_LOAD_CONFIG]
               [--kernel-save-config KERNEL_SAVE_CONFIG]
               [--kernel-localversion KERNEL_LOCALVERSION]
               [--kernel-menuconfig] [--rootfs-source ROOTFS_SOURCE]
               [--rootfs-source-sha512 ROOTFS_SOURCE_SHA512] [-o OUT]
               [-l LOG_FILE] [-v]
               l4t_path
tegrity: error: unrecognized arguments: --menuconfig

Many thanks for your great support!

37

Yw. Thank you for testing. You helped find some bugs.

I believe the option is now --build-kernel --kernel-menuconfig. I don’t remember if --kernel-menuconfig implies --build-kernel.

You can do a tegrity --help for full usage.

So:

sudo tegrity -v --build-kernel --kernel--menuconfig ...

It will change in the next release as well. The issue is the main script, ‘tegrity’, runs the main functions in the other scripts (eg. ‘tegrity-kernel’, but those have many more options than what ccan reasonably be presented in one script. Nobody likes a wall of options, even with grouping. VLC is particularly bad.

Instead, the new version of the main script will take a config file and there will be a GUI to generate that (or you can generate/modify one with a template and a text editor). The individual script options will probably not change too much, and you’ll still be able to run them step by step.

For example, in your case, if you are only customizing the kernel, you can just use ‘tegrity-kernel’ to make a custom kernel and ‘tegrity-image’ to make a SD card image (or just use the Nvidia script it calls).

38

Hi mdegans

You’re right, with your command the process is starting and the menuconfig is shown.
I was able to select RAID support but after a few minutes I got the following error:

running: /home/marco/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/apply_binaries.sh -t -r /home/marco/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/rootfs
Using rootfs directory of: --
install: missing file operand
Try 'install --help' for more information.
Traceback (most recent call last):
  File "/usr/local/bin/tegrity", line 11, in <module>
    load_entry_point('tegrity==0.0.3', 'console_scripts', 'tegrity')()
  File "/usr/local/lib/python3.6/dist-packages/tegrity/__main__.py", line 139, in cli_main
    return build(**kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tegrity/__main__.py", line 69, in build
    fix_sources=rootfs_source,
  File "/usr/local/lib/python3.6/dist-packages/tegrity/rootfs.py", line 183, in main
    apply_binaries(rootfs, target_overlay=target_overlay)
  File "/usr/local/lib/python3.6/dist-packages/tegrity/rootfs.py", line 163, in apply_binaries
    tegrity.utils.run(command, cwd=l4t_path).check_returncode()
  File "/usr/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)

subprocess.CalledProcessError: Command '['/home/marco/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/apply_binaries.sh', '-t', '-r', '/home/marco/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/rootfs']' returned non-zero exit status 1.
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 67, in apport_excepthook
    binary = os.path.realpath(os.path.join(os.getcwd(), sys.argv[0]))
FileNotFoundError: [Errno 2] No such file or directory

Original exception was:
Traceback (most recent call last):
  File "/usr/local/bin/tegrity", line 11, in <module>
    load_entry_point('tegrity==0.0.3', 'console_scripts', 'tegrity')()
  File "/usr/local/lib/python3.6/dist-packages/tegrity/__main__.py", line 139, in cli_main
    return build(**kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tegrity/__main__.py", line 69, in build
    fix_sources=rootfs_source,
  File "/usr/local/lib/python3.6/dist-packages/tegrity/rootfs.py", line 183, in main
    apply_binaries(rootfs, target_overlay=target_overlay)
  File "/usr/local/lib/python3.6/dist-packages/tegrity/rootfs.py", line 163, in apply_binaries
    tegrity.utils.run(command, cwd=l4t_path).check_returncode()
  File "/usr/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/home/marco/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/apply_binaries.sh', '-t', '-r', '/home/marco/nvidia/nvidia_sdk/JetPack_4.3_Linux_P3448/Linux_for_Tegra/rootfs']' returned non-zero exit status 1.

Do you know how to proceed?

Many thanks & best regards!

39

Looks like the kernel built successfully.

For now, try navigating to your Linux_for_Tegra folder and running:

./apply_binaries.sh -t -r rootfs

To install the kernel and modules to the rootfs directly using Nvidia’s script.

Then, if that works, run:

sudo tegrity-image -v -o sdcard.img .

The dot on the end represents the current path. If you run it from outside the Linux_for_Tegra folder, replace it with the path to Linux_for_Tegra instead.

40

It doesn’t work:

# sudo ./apply_binaries.sh -t -r rootfs
Using rootfs directory of: --
install: missing file operand
Try 'install --help' for more information.