[CLONE ISSUE] Between [TX2 R28.2.0] and [TX2 R28.2.1] [Where can get -JetPack 3.2 with TX2 R28.2.0]

Hi. Good day to you.

Currently I have 10 sets TX2 unit.

4 sets existing TX2 board is version B02 and they work well.

1 new set B-02 TX2 board and 5 new sets D-01 TX2 board

My existing B02 TX2 board has the following version

[i]nvidia@tx2:~$ head -n 1 /etc/nv_tegra_release

R28 (release), REVISION: 2.0, GCID: 10567845, BOARD: t186ref, EABI: aarch64, DATE: Fri Mar 2 04:57:01 UTC 2018[/i]

And I have used my exising B-02 TX2 board to make a clone image. (L4T 28.2.0 as above shows)
->sudo ./flash.sh -r -k APP -G my_backup.img jetson-tx2 mmcblk0p1

After that, I flash this image to my 6 new TX board,the issue happens, which as follow:

  1. Done a full flash by useing jetpack 3.3 for the new [b]B-02 TX2 board[/b]
    

1.1 flash my existing clone image to this new B-02 TX2 board
->sudo ./flash.sh -r -k APP jetson-tx2 mmcblk0p1
1.2 The new B-02 TX2 board works normally.

  1. Done a full flash by useing jetpack 3.3 for the [b]1st[/b] new [b]D-01 TX2 board[/b]
    

2.1 flash my existing clone image to this 1st new D-01 TX2 board
->sudo ./flash.sh -r -k APP jetson-tx2 mmcblk0p1
2.2 The 1st new D-01 TX2 board graph driver can’t work normally and seems CUDA also got problem.

  1. Done a full flash by useing jetpack 3.3 for the [b]2nd[/b] new [b]D-01 TX2 board[/b]
    

3.1 flash my existing clone image to this 2nd new D-01 TX2 board
->sudo ./flash.sh -r -k APP jetson-tx2 mmcblk0p1
3.2 The 2nd new D-01 TX2 board has exactly same problem as the 1st new D-01 TX2 board

Now I am not sure which part is wrong. can anyone help me about that?

Is that because board B-02 image can’t be used for D-01? How can I slove it?

Appreciate for your time and helping!

Just to clarify one detail, can you confirm that all clone recipient boards were initially flashed with the same R28.2 which runs on the clone donor? I ask because it is necessary for a board receiving a clone to have the same release version as the source.

If this is true then you can probably consider the next step in the issue to be comparison of device tree requirements. I don’t know if board revision level would change what the device tree needs to be, but the flash.sh script does use some different files depending on revision. Presumably the rootfs itself is insensitive to board revision.

Hi Linuxdev, Thanks for the helping. To answer your clarify:

I can’t find initial R28.2.0 for TX2 now.

So for my initial flashing, I have tried both Jetpack 3.3 and Jetpack 3.2.1, both are R28.2.1, and I flash it for both B-02 board and D-01 board
In the end, only B-02 works.

I have just reflash the Jetpack3.3 R28.2.1 to the Board B-02, then I flash my clone image [R28.2.0] to the same board,
same issue happen as Board D-01

I think maybe I need R28.2.0 for my new TX2 first…

  1. May I know where can I get the R28.2.0 for TX2?
  2. When I do the full initial flashing, I run the Launcher under _install jetpack folder, and only choose File system and OS & Drivers for the target. Is that correct?

Thank you.

Just now I have tried JetPack 3.2 which has R28.1.0.

I flash JetPack 3.2 with R28.1.0, it can boot up. After that, I have flashed my R28.2.0 to the TX2, the TX2 can’t boot up even.

When flash JetPack 3.2.1/3.3 with R28.2.1, then flashed my R28.2.0, the TX2 can boot up, but with many issues, graph card driver issue, CUDA issue etc…

I can’t find R28.2.0 within archives but R28.1 : https://developer.nvidia.com/embedded/jetpack-3_1
However, may be you could either approach rsync method, or reconstruct the environment manually at a new-release-flashed device?
Is that something that can be implemented with manual efforts on a flashed Jetson?

Hi Andrey, I have tried R28.1, the TX2 can’t boot up after R28.1 → R28.2.0 (my backup clone image)
To reconstruct on a new base (R28.2.1) is a bit difficult. T_T

And from the hide link, https://developer.nvidia.com/embedded/jetpack-3_2

I have download Jetpack-L4T-3.2_b157, however, it’s still 28.1.0.

Does anyone have the Jetpack 3.2 for TX2 with R28.2.0?
###############################################################################

L4T BSP Information:

R28 (release), REVISION: 2.0, GCID: 10567845, BOARD: t186ref, EABI: aarch64,

DATE: Fri Mar 2 04:57:01 UTC 2018

###############################################################################

Appreciate for your helping.

The L4T version listings is here (you probably need to go here, login, and then hit the URL again since forwarding doesn’t work right):
https://developer.nvidia.com/embedded/linux-tegra-archive

Then click on the relevant R28.1, R28.2, R28.2.1, so on.

I’d have to sit there and experiment to know for sure, but here are some tips which might speed it up some.

The “sudo ./apply_binaries.sh” step adds the content which is NVIDIA-specific (other than boot content). Your cloned system can afford to have this applied to it without losing your own personalized content, and this might be the place to start.

So I assume you have a safe clone somewhere and I’m not suggesting altering your reference copy. This is an example using loopback mounting (it assumes you have the raw image and not just the sparse image).

Start by mounting the image on loopback over the top of the “rootfs/” content. For example:

cd /where/ever/it/is/Linux_for_Tegra
# Assumes you have a clone with the name system.img.raw, but that doesn't really matter so
# long as it is ".raw" actual file format. If you only have the sparse image you are out of luck.
sudo mount -o loop /some/where/else/clones/system.img.raw ./rootfs/
sudo ./apply_binaries.sh
# Now flash without "reusing" the rootfs...the rootfs is already your clone, and in addition
# to the apply_binaries.sh content change the bootloader content will also change, e.g., the
# kernel itself can be changed. TX2 is in recovery mode with micro-B USB.
sudo ./flash.sh jetson-tx2 mmcblk0p1
# Or with logging...gawk to reduce the progress bar logging:
sudo ./flash.sh jetson-tx2 mmcblk0p1 2>&1 | gawk '{gsub("[0-9][0-9]+[/][0-9[0-9]+ bytes sent..",".");print}' | tee log_flash.txt

This should give you what is essentially your original clone, but the boot infrastructure of whichever flash.sh driver package version you used. I wouldn’t recommend this across major version changes, but this should be fine between R28.2, R28.2.1, and R28.3.

Note that when you mount something on “rootfs/” that the content in “rootfs/” is unmodified, but will be hidden and is replaced by the mounted content. Unmounting (umount) of the loopback image would restore the “rootfs/” original content visibility. You could instead recursively rm everything in “rootfs/” and do a recursive file copy with options for preserving numeric IDs and permissions, but this actually takes longer and is more likely to be error prone. The loopback mounted image will be modified, but only for boot and NVIDIA-specific hardware files.

Hi Mr.linuxdev, many many thanks for your advice and help. It looks work for my case.

First, I didn’t try the R28.2.0 from https://developer.nvidia.com/embedded/linux-tegra-archive. Becasue from the nvidia website, it shows for TX1 only and it’s Tegra210(210for TX1 maybe?) and my TX2 board is Tegra186

Then, I use your suggestion → loopback mounting.

  1. I backup my existing R28.2.0 imagefile and keep the .raw image
  2. Then follow your step to do the loopback mounting to my previous failed board (the failed board has initial flash R28.2.1, and flashed my R28.2.0 image)
  3. After loopback mounting flash, the failed board boot up successfully and all my workspace and program can run as normal.
  4. I have check the successful board, the L4T version is 28.2.1 and the work space is same as my R28.2.0 image.

Appreciate your helping!

Besides, I have 2 questions want to confirm with you about this loopback mounting flash method.
Q1: For the brand new board, Do I still need to do the initial full flash by using Jetpack3.3(with R28.2.1) Before I do the loopback mounting flash?

Q2: I am using Auvidea-j130 carrier board. Previously, my clone image has already install the Auvidea kernel and dtbs via Jetpack by replacing the Jetpack/Linux_for_Tegra/ dtb, kernal files then run the ./apply_binaries.sh.
In future, if I want to change and use another new carrier board, can I do as the following steps?

  1. Get my existing image.raw
  2. loopback mount the raw image
  3. ./apply_binaries.sh
  4. copy and replace new carrier board file to the exising Jetpack/Linux_for_Tegra/ dtb, kernal files
  5. ./apply_binaries.sh again
  6. ./flash.sh jetson-tx2 mmcblk0p1
    Will that work?
    Or it will still go back to my old carrier board because the .raw clone file is still installed my old carrier board So I need to creat another flesh image with new carrier board files.

I think I have a little confuse betwwen the

  1. ->loopback mounting flash
  2. ->normal clone ./flash.sh -r -k APP as my did before
  3. initial full flash
    If you could have time, could you briefly explain or sent some link to me for these flash/clone part? I want to know more about it.

Many thanks and Have a nice weekend!

you may like to check additionally if your mac address of devices differ, as otherwise you will have to manually adjust them, in my opinion.
I did not use that particular way, but it seems that if you are restoring the raw image of one device to another device [ which is not necessarily the case, as you are probably restoring a raw image to the very same device], then in the network context you could come up with two devices both having the same mac address for network adapters.
Thank you for sharing your success.

Hi Andrey, Thank you for the suggestion,

I have used loopback mount flash 2 device, and just checked, they have different mac address.

Just some general information you might find useful…

A clone is an exact bit-for-bit copy of the rootfs partition as a whole. This means it includes the ext4 file system and is not just a copy of files. Flashing a clone implies flashing an image (the ext4 file system is built in…the flash software does not need to understand ext4 as it only treats this as binary data).

During a normal flash an empty file is created which is the exact size of the rootfs partition. That file is covered with loopback, and because of this the file can be treated exactly the same as a partition. That loopback file is formatted with ext4. Once that is done the files of the “rootfs/” directory have some boot-related content added (which isn’t a lot of content), and then copied into the loopback file. This file becomes “bootloader/system.img.raw”. If you create a clone, and the clone is from a freshly flashed Jetson which has never booted, then the clone is an exact match of the “system.img.raw” down to the last bit.

During flash the “system.img.raw” is used to create a “sparse” version of this (a sort of poor man’s compression for lower size). That file becomes “bootloader/system.img”. This file is then flashed to the actual TX2. The TX2 itself automatically understands a sparse image versus raw image, and will unpack the sparse image back into the full size raw image before writing it to eMMC.

If you were to replace “system.img” by overwriting it with the larger “system.img.raw”, then the resulting flash would be exactly the same other than taking much longer (sending a couple of GB over USB2 is much faster than sending 16GB over USB2).

To reiterate, when you flash normally an image is created from “rootfs/” content. The “system.img” and “system.img.raw” files remain after flash. If you were to then use the “-r” option on command line flash to “reuse” the image, then “rootfs/” would be ignored and whatever sits at “bootloader/system.img” is flashed.

Placing a clone at “bootloader/system.img” and then reusing the image (the “-r” option) causes the clone to be stored directly on eMMC without modifying it. Placing the clone in “rootfs/” instead, and not reusing the image (not using “-r”) causes nearly the same thing…only some of the boot content is freshened to this L4T driver package release, but otherwise it is a 100% exact match for everything else.

Summary:

  • Normal flash generates an image from "rootfs/" which is populated by the Ubuntu sample rootfs.
  • Normal flash with a clone in "rootfs/" generates an image which is populated by your clone.
  • Flash which reuses the image depends on the image previously created and does not know what the image source was.
  • Flash which reuses the image, where the image happens to be from a clone, flashes that image and does not know, nor does it care, that the image is from another Jetson and not freshly generated from a sample rootfs.

Something @Andrey1984 was mentioning is important in some cases because a clone copies everything. Passwords are copied, network setup is copied, so on. This is probably what you want, but if you are going to a new TX2, then something dependent upon serial numbers or MAC addresses might be looking for a serial number or MAC address from the previous module. This part would have to be edited to work again on the new module. The udev system for renaming certain interfaces sometimes gets customized to rename a network based on the hardware MAC address…and that udev rule would fail on a new module until the MAC address is edited. Most of the time MAC and serial numbers are not used and so it won’t matter…in the few cases where MAC address or a serial number is used and something odd doesn’t work, then it is worth investigating udev for whether MAC address/serial number was hard wired in some customization. Assuming your network works as it should, then this won’t be an issue and MAC customization did not exist on the prior board.

As an example, imagine if you have a firewall rule which applies to the MAC address. The firewall rule would become inert on a new module until the MAC is fixed.