Jetson won't load up after configuring SPI functionality

I was following the walkthrough in https://elinux.org/Jetson/TX2_SPI, getting down to the section labeled “Enabling the new DTB”. The

sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1

command didn’t take, so I skipped it and when to the next part of updating the extlinux.config file. I updated the line that has “/boot/dtb/tegra186…” in it, but after restarting it the screen comes up with:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
bash:

I have tried restarting the Tx2, to no avail. I went back and tried to see if there were any steps that I missed that could have been crucial, and realized that it states “Do not update the DTB by below for r28. r32 and later release. As in Jetson/TX2 DTB, enable FDT in /boot/extlinux.conf”. This wording is slightly confusing, and truth be told I’m not sure what r28, r32, etc. that I am using, and so I don’t know if the issue is that I updated this line incorrectly. Is there a way to at least get back into the .config file for me to try and fix it, or is there anyone who has had this issue/one similar and knows how to fix it?

Only the earlier releases used the “/boot” dtb file (and adding the FDT entry in extlinux.conf could cause boot failure). The kernel-dtb partition, if you began flashing, could have been erased or corrupted (this is the device tree used in boot on more recent releases).

Note that drivers usually provide a device special file in “/dev”, and that most access to those files is ordinary file style read/write software. Those files though are not real files, and are instead communications directly with drivers in the kernel. The IOCTL is a custom interface which individual drivers can choose to implement as they see fit, e.g., maybe a serial port file can be read or written, but to change its speed you’d use an IOCTL call. The failed IOCTL indicates the driver and hardware no longer match the attempted setting to that file. Device tree failure or loss of a driver would account for this (without the device tree the driver won’t be able to find some hardware…“the lights are on, but nobody is home”).

If you changed the extlinux.conf to point at a device tree in “/boot” with the FDT entry, then you’ll need to remove that. If only the kernel-dtb partition is corrupt, then you should be able to run the same command using the correct device tree to put the partition back in place and it would start working again. This latter may be harder than it sounds though since it sounds like you don’t know which device tree file is correct.

You could simply clone the rootfs (always a good idea with developing if the partition needs backup anyway) and then reflash while telling it to use the clone for the rootfs (it’d be a new flash of all partitions, even the rootfs, but it’d happen that the rootfs being flashed is an exact copy of the existing rootfs). Note that cloning takes a lot of time and disk space (I wouldn’t start on a host PC with less than about 35GB+ of spare space…better yet at least 40GB+). Clone commands also depend on the release currently flashed (and restore depends on using that same release if a cloned partition is to be put back in).

Was extlinux.conf edited? If not, then the dtb file change in “/boot” is unlikely to be related to the issue. Any progress at all into writing into kernel-dtb (followed by failure) is very likely to be a problem.

Ok so the walkthrough mentioned above is semi-out-of-date as it builds kernel 4.4. The GIT now downloads kernel 4.9, and so some of the steps didn’t work/worked slightly differently than how the walkthrough entailed. I would like to warn you ahead of time that this is a big mess as I misunderstood/misread a few of the steps. I skipped the “./copyImage.sh” step as I misread the walkthrough and thought I had already done that step. I didn’t think this was an issue as the spidev.ko file was already made in the appropriate drivers folder. I was able to install the device tree, but when I went to “update the device tree”, there wasn’t any file called “myTX2DeviceTreeSource.dts”, but there was a file “extracted_proc.dts” which was the only .dts file there. This .dts file seemed to fit the format for what the walkthrough wanted me to edit. Again no problems until the Enabling the DTB section, where the ./flash.sh command wouldn’t do anything, so I skipped it and thought it was a product of the walkthrough being outdated. I did edit the extlinux.conf with the line that in hindsight they specifically said to only do for r28, which seems to be the problem. When the bash shell comes up, I can’t access the /boot directory that the file was in, nor am i able to really edit anything (side note I’m familiar with linux commands but not so much with bash shell scripts. I know it’s pretty much the same but it appears as if the selection for commands is MUCH less than a normal terminal).
If you can provide any tips or suggestions I would greatly appreciate it. This is for a major project and it would be a huge problem if I can’t get the Jetson back up and running.

If you have the time and disk storage space to run a clone, then you are probably set. Before I tell you about that there are some descriptions of how flash works which would be useful to keep in mind (the extra I’m starting with isn’t strictly necessary, but you’ll understand what I’m showing you as a workaround to changes made).

When a Jetson is flashed from scratch there is a “Linux_for_Tegra/” directory. SDK Manager will download this for you if you have run this. Typically, for a TX2, this will be located using the name of the release:

~/nvidia/nvidia_sdk/JetPack_<some version like 4.3>_Linux_P3310/Linux_for_Tegra/

(the P3310 is a reference to the TX2)

Within this there is a “rootfs/” subdirectory, and this is populated by unpacking the sample Ubuntu rootfs with sudo. Some NVIDIA-specific hardware drivers added by running the “apply_binaries.sh” script with sudo (JetPack does this for you, but it can be done manually). This is almost an exact copy of what the final image will contain.

I say “almost” an exact copy because during the flash various options will be used to decide on content to copy into “/boot”. Those options are specific to the module/carrier board and current release. This includes the kernel “Image” file.

To illustrate, if you’ve flashed, then verify matching checksums:

cd /where/ever/it/is/Linux_for_Tegra
sha1sum bootloader/Image rootfs/boot/Image

…the checksums should match. If you have a clone, and this checksum does not match the Linux_for_Tegra content, then you’ve probably updated the base kernel Image (which would be a candidate for putting back in place using the “bootloader/Image” file).

Upon beginning a flash all of that content becomes a binary ext4 image:

bootloader/system.img.raw

…which is a bit-for-bit exact copy of what the rootfs partition has in it after the flash.

If you were to edit a file in “rootfs/” prior to flash, and if that file is not part of the boot content being edited based on options at flash time, then the change will persist.

If you flash with the “-r” option, then the steps to use “rootfs/” to create “system.img.raw” are skipped, and only the current content which is already there is used. If you are using the stock image, then after all there is no reason to create the same “nearly 30GB” file again, over and over, only to have the same exact content.

That system.img.raw file is used to create a smaller “sparse” file, “bootloader/system.img”. During the flash any file with name “bootloader/system.img” is put in the rootfs partition (a sparse file is unpacked in a way similar to compression, so it is still an exact match for system.img.raw). You could in fact copy system.img.raw into system.img, specify “-r” to flash.sh, and the only difference in final result would be a longer time to flash since the file is bigger (but there is no unpacking step since it is raw instead of sparse).

If you were to use the “-r” option, and replace “bootloader/system.img” with your cloned image, then the original clone would exactly install after the flash. The surrounding content, e.g., partitions with a device tree, would be updated with the correct content (so long as the “Linux_for_Tegra/” device tree content had not been manually edited by some prior customization). If the original filesystem had no issues, then the system would just start working again.

If instead of the unpacked sample rootfs being in “rootfs/” you had a loopback mounted clone from the original system, and you were to flash without the “-r”, then a new system.img would be generated. That image would be a nearly exact copy of the original rootfs you cloned. The only change would be that any “/boot” content from flash.sh options would be copied into the clone, and then the system.img produced. Had your Image or other files been edited, and had the flash options wanted to put their own version in place, then your image would be all that you started with…except it would then have the correct Image or other “/boot” content.

Important: If your clone is from a particular release, e.g., from “R32.1”, then you must flash with the same JetPack/SDKM release as that which originally installed the R32.1 image. The non-rootfs partition content has dependencies.

On the more recent releases you can clone like this from the “Linux_for_Tegra/” directory if the TX2 has the micro-B USB connected and is in recovery mode (warning, make sure you have at least about 35GB+ of free disk space on the host PC):

sudo ./flash.sh -r -k APP -G my_backup.img jetson-tx2 mmcblk0p1

The above produces a sparse file, “my_backup.img”, and a raw file, “my_backup.img.raw”. Delete the smaller “my_backup.img” (sparse) file since you cannot modify that version (nor examine it).

Be careful to not overwrite your original my_backup.img.raw, but also note that if you copy this file, then you need another 35GB or so of space (there is some temporary file activity going on, so even if the file is 28GB I wouldn’t advise using less free space).

To illustrate loopback mounting:

sudo mount -o loop ./my_backup.img.raw /mnt
ls /mnt
sudo umount /mnt

If we were to put this into “rootfs/” and intentionally refresh the “rootfs/boot/” content, we’d loopback mount that there and generate a new image, combined with flash. The result would be everything except the “boot” content being refreshed with valid content (including kernel-dtb partition and Image in “/boot”):

sudo mount -o loop ./my_backup.img.raw rootfs/
# Verify you see stuff there:
ls rootfs/
lsblk -f rootfs/
# Now connect the Jetson in recovery mode plus the micro-B USB cable, and flash:
sudo ./flash.sh jetson-tx2 mmcblk0p1

Now your original image is back in place as a 100% exact copy of any content not part of “/boot”, and all supporting partitions are also restored. Any previous additional packages you’ve added via JetPack (e.g., CUDA) will still be there. Any custom programming you’ve created (as long as it isn’t in the kernel binary Image) will still be there. Network setup will still be there, passwords will still be valid, so on.

Incidentally, as large as the my_backup.img.raw file is it will take several minutes (maybe part of an hour) to make a single copy. I tend to “bzip2 -9 my_backup.img.raw” when I’m storing it, and compression itself will take around an hour for a single file depending on host PC specs.

Once the flash is done you would also want to unmount the rootfs mount:

sudo umount ./rootfs/

For reference, if the original my_backup.img.raw is saved elsewhere, but the new boot works, then you might save the rootfs loopback mount image instead of the original…the “/boot” content will now be valid.

Had you just wanted to use my_backup.img.raw directly then you could skip any loopback mounting, copy it to “system.img”, and then flash…but with the “-r” option so system.img does not get overwritten:

sudo ./flash.sh -r jetson-tx2 mmcblk0p1

Once you have loopback mounted onto “rootfs/” and updated the “/boot” content there is no reason to generate another new image…that loopback image would have been directly edited to contain “/boot” changes, and thus is now directly usable as “bootloader/system.img” as a bit-for-bit exact match.