Does anybody know how to create a RAID10 with 4 USB devices on Jetson Nano?
The following is my way with the error:
# sudo apt-get install mdadm -y # sudo mdadm --create --verbose /dev/md0 --level=10 --raid-devices=4 /dev/sda /dev/sdb /dev/sdc /dev/sdd
mdadm: layout defaults to n2
mdadm: layout defaults to n2
mdadm: chunk size defaults to 512K
mdadm: partition table exists on /dev/sda
mdadm: partition table exists on /dev/sda but will be lost or
meaningless after creating array
mdadm: partition table exists on /dev/sdb
mdadm: partition table exists on /dev/sdb but will be lost or
meaningless after creating array
mdadm: partition table exists on /dev/sdc
mdadm: partition table exists on /dev/sdc but will be lost or
meaningless after creating array
mdadm: partition table exists on /dev/sdd
mdadm: partition table exists on /dev/sdd but will be lost or
meaningless after creating array
mdadm: size set to 244448256K
mdadm: automatically enabling write-intent bitmap on large array
Continue creating array? y modprobe: FATAL: Module md_mod not found in directory /lib/modules/4.9.140-tegra
mdadm: Fail create md0 when using /sys/module/md_mod/parameters/new_array
mdadm: unexpected failure opening /dev/md0
I donât have a lot of ubuntu-knowledge , I need your help!
Many thanks for your support!
Apparently, the kernel isnât compiled with the âmdâ driver built in, and the module to support the âmdâ device is also not included by default.
This seems kind-of annoying â I think any linux system should have MD and LVM built into the kernel, but apparently NVIDIA thought different.
You will need to follow the instructions to re-build the kernel, OR the instructions to build a new module for the existing kernel, to build the md_mod module to load into your kernel. Unfortunately, because file system mounts happen early, that module generally needs to be available in the early boot stage, not just user-loaded once the system is up. Which is why I recommend building the kernel with the module static-linked ("configure âyâ instead of âmâ)
First off all, many thanks for your helpful feedback!
I think, I understand you, but honestly Iâve no idea how to recompile a linux kernel which contains MD and LVM.
What exactly do you mean with building the kernel with the module static-linked ("configure âyâ instead of âmâ)
and where can I found the instructions?
When you build something into the Linux Kernel, it can generally be built in to the kernel, or included as a module. If you need to access somethign at early boot time (like to boot off something), itâll need to be a âyâ rather than an âmâ.
Basic instructions from nvidia are here but forewarning: this process is not easy.
You should be able to run
âmake menuconfigâ (or xconfig, if you prefer) instead of modifying the config file by hand after you run
âmake ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_defconfigâ
(between steps 3 and 4)
Then you navigate to these bits and change the options you need for raid to âyâ
Once youâve built and booted the system, youâll then need to âsudo apt install mdadmâ, and then set up raid as usual.
Edit: for those new to the thread, there are some scripts now to do all of this automatically:
FYI, answering âyâ puts the driver or content in the actual kernel Image. Ansering âmâ puts the content in a module, which is a separate file. Often modules are a better choice, but items needing access during boot, and certain invasive options, typically need âyâ. In cases where you canât integrate a feature required during boot (a.k.a., features you canât use âyâ with), then you have to resort to a lot of trouble creating an initial ramdisk (initrd). Just using âyâ is a lot easier, but you might need to rebuild any modules (when you change the base Image, sometimes modules which plug into the Image no longer work, or have quirks).
The Image is part of what determines your âuname -râ output (the suffix is usually from the config item CONFIG_LOCALVERSION, and on a Jetson, usually CONFIG_LOCALVERSION=â-tegraâ). Modules are always searched for at:
/lib/modules/$(uname -r)/
If add a module, then things are usually a very straightforward file copy. If you change Image, but keep âuname -râ the same, then possibly all modules will work as expected from the old â/lib/modules/$(uname -r)/â, but not always. The Image file is also a file copy to â/boot/Imageâ, but be sure to save the original Image prior to putting in a new Image.
FYI, I suggest copying â/proc/config.gzâ somewhere as a reference to the original running systemâs config prior to working on a kernel. A reference copy can be very useful.
Seems youâre missing libncurses. You may check which version is available with:
sudo apt-get update
sudo apt search libncurses
#depending on what is available for your distro, install with something like:
sudo apt-get install libncurses5 libncurses5-dev
If your goal is: increase system reliability A simple RAID 1 is enought.
If your goal is: increase storage A NAS mouted over SSH has more sense, if you think that 1Gbit/s is not enought you can plug more ETH network cards over USB3 and bonding them (obviously, also NAS needs more than 1 ETH port).
yes, Iâve missed libncurses.
So after the installation of it, I was able to invoke menuconfig.
But I get an other error after step 8: If Iâm going to reboot the system, It hangs and I donât get any output. Do you know why?
Following my actionplan:
I donât have time to respond at length as I just got back from vacation and have a lot to catch up on, but it looks as if you may be missing some steps from Nvidiaâs instructions to copy the appropriate files.
If you do steps 4-8 on Nvidiaâs instructions after the menuconfig and then run the create nano SD card script, you should end up with a bootable system. If itâs your dev system, rather than the nano, that isnât booting, that shouldnât be related. Fwiw I donât see reboot in the steps. You shouldnât have to reboot your dev machine at all during the process.
@mdegans: Many thanks for your feedback. Yes youâre right, I thought I was finish at step 5.
Now, Iâm stucking with step 7 â I donât know what to write for the INSTALL_MOD_PATH.
Do you know it?
In addition, which create-nano-SD-card-script do you mean?
sudo make ARCH=arm64 O=$TEGRA_KERNEL_OUT modules_install INSTALL_MOD_PATH=<whatreverthepathisto>/Linux_for_Tegra/rootfs/
You donât actually replace âINSTALL_MOD_PATHâ with anything. Make sure if you go back after the fact and try to continue that you set TEGRA_KERNEL_OUT again as in step 1.
The create-nano-sd-card script can be found in the Linux_for_Tegra folder wherever you installed sdkm. Itâs a script that runs flash.sh with the proper parameters to create a nano sd card. When you get to that point, let me know, or run the script with --help.
Ah okay, so you mean I have to install first sdmk to get this folder âLinux_for_Tegraâ?
Then I need to execute the step 5 and 6 from âBuilding the NVIDIA Kernelâ again, because I did it in the wrong way.
Until point 4 (inlcuding the menuconfig), I understand it and it works.
But Point 5, 6 and 7 I dont know â<release_packagep>/Linux_for_Tegra/kernel/Imageâ, what is it?
Point 8 is optionally.
@mdegans: You wrote âYou donât actually replace âINSTALL_MOD_PATHâ with anythingâ, really?
I need help to get raid support on my jetson nano.
SDK Manager is capable of working with more than one Jetson system. That ârelease_âŠâ is a version number for the module/carrier board. If youâve used SDK Manager for flashing only a Nano, then there will be only one subdirectory to â~/nvidia/nvidia_sdk/JetPackâŠLinuxâŠ/Linux_for_Tegra/â.
Remember to examine CONFIG_LOCALVERSION (see post #7). If you replace an Image, then you must also replace every single moduleâŠunless you set up CONFIG_LOCALVERSION to match the previous Image.
So, I need to use SDK Manager to compile the linux kernel?
I didnât used SDK Manager to get the linux on my sd-card, I used the following command on another system:
That command is is to flash a binary system image to the SD card. What you need to do is to compile (make binary) a new kernel from source, add it to the appropriate folders inside wherever you installed SDK managed, run the SD card creation script to make your own binary image, and then flash that to the SD card like you did before with dd. In your case, yes, you must install SDK Manager, pick the options for your device, and obtain the kernel sources as is outlined in Nvidiaâs instructions.
If youâre not familiar with compiling the Linux kernel, it may be difficult. Nvidiaâs instructions seem to be intended towards people who have already done it before.
When I have time I plan to write a script to set things up, prompt the user with menuconfig, and spit out a flashable image. I will try to get around to that by December, but no promises. I need to re-run through the process from beginning to end on a new user, take notes, write it, and fix any bugs first. In the mean time, please keep trying. This isnât an easy task.
Yes. That command should be â⊠INSTALL_MOD_PATH=some/path/here âŠâ Like in the code block in post #14
It should be the path to rootfs/lib/modules where you want your modules installed. By defining INSTALL_MOD_PATH, youâre telling the kernelâs scripts where you want your modules installed. The modules will need to match the kernel (âImageâ file) version or the system will probably not boot.
The ârootfsâ folder, if youâve found it, becomes â/â on the nano when the SD card creation script makes the image. Any modifications or files added there will be changed on the final image as well. For example, if you wanted to add something in /etc on the nano it would go in rootfs/etc on your workstation.