Error: Return value 1 Command tegradevflash_v2 --pt flash.xml.bin --create Failed flashing t186ref

I am beginning to see more frequent occurrences of problems flashing my fleet of AGX modules. This “Error: Return value 1 Command tegradevflash_v2 --pt flash.xml.bin --create Failed flashing t186ref.” seems to be the most frequent error. I initially reported 10 units but now I am at 15.

Hi andrew.k.lore,

Are you using the devkit or custom board for AGX Xavier?
What’s your Jetpack version in use?

Could you help to provide the full serial console log for further check?

Do you mean that not all boards with this flash issue?

I am using a ConnectTech board. JetPack version 4.3, no we cannot upgrade. I will run it again and pull the serial logs.

Two of the boards hung at ‘tegrarcm_v2 --isapplet’.”

Nothing is displayed in the serial console log. I am including text from the output on the host computer. The serial connection is functioning properly, I checked with another stack.

Flash_Fail.log (44.2 KB)

I was able to capture this from a failed flash. It’s the serial output you requested earlier.

Flash_Fail_Serial.log (6.75 KB)

JP4.3 is quite an old release.
Is there any reason for you to not update to latest JP4.6.3(R32.7.3) or JP5.1.1(R35.3.1) for AGX Xavier?

It seems you are using the following command to flash the board.

sudo ./flash.sh -r cti/xavier/rogue mmcblk0p1

Could you try to use the following command instead?

sudo ./flash.sh cti/xavier/rogue mmcblk0p1

Yes, as I stated we cannot upgrade at this time. There are multiple dependencies that have not been resolved to allow an upgrade.

No, we cannot have the system.img regenerated every time. We have a production image we send to the stacks. There is nothing wrong with the image as I have used it to clone over 400 unique stacks, this is not an exaggeration. I can try this as a troubleshooting step but it’s not going to be an acceptable long term solution.

No, we cannot have the system.img regenerated every time. We have a production image we send to the stacks. There is nothing wrong with the image as I have used it to clone over 400 unique stacks, this is not an exaggeration. I can try this as a troubleshooting step but it’s not going to be an acceptable long term solution.

No change in behavior.

That is just to help for the debug.

Do you have the successful flash log?

Please also provide the board config file (cti/xavier/rogue.conf) for further check.

You should have received the flash failures in another response.

I have attached the successful flash logs and the configuration requested. I had to reflash a stack to get these so the image should be sound.

flash_success_host.log (52.1 KB)

flash_success_serial.log (39 KB)

rogue.conf (186 Bytes)

It seems the issue relating to partition table of secondary_gpt.

In Flash_Fail_Serial.log

[0022.654] W> Cannot find any partition table for 00010003
[0022.654] E> NV3P_SERVER: Failed to initialize partition table from GPT.

In Flash_Fail.log

[  13.5807 ] 000000000d0d0001: o initialize partition table from GPT.

Do you modify the following content in flash_t194_sdmmc.xml?

        <partition name="secondary_gpt" type="secondary_gpt">
            <allocation_policy> sequential </allocation_policy>
            <filesystem_type> basic </filesystem_type>
            <size> 0xFFFFFFFFFFFFFFFF </size>
            <file_system_attribute> 0 </file_system_attribute>
            <allocation_attribute> 8 </allocation_attribute>
            <percent_reserved> 0 </percent_reserved>
            <description> **Required.** Contains secondary GPT of the `sdmmc_user`
              device. </description>
        </partition>

Or modify anything relating to the partition?

Does this flash failed issue happen occasionally?

No, there were no modifications.

I was able to flash other modules before and after with the same environment. Refer to the other logs I sent last.

The units are in service in the field until we can no longer detect a heartbeat. They are replaced and returned to my facility, where we boot them without modification and observe behavior. These stacks never fully boot, so a reflash is attempted and then we encounter the error listed. Same error for most of them. Some are completely dead and some have kernel panics. We resolved the kernel panics with a reflash but nothing we attempt in house can restore the others to working conditions. I now have a growing wall of modules that cannot be returned to service.

I only happens on specific modules, and no, re-attempting to reflash does not alter results no matter how many attempts.

Is it possible to put those modules on devkit to test instead of your board?

I can give it a try.

Yes, please put it on devkit, flash it with pure sdkmanger + jetpack.

If this module is old one that ever got used before, flash the same version as that time.
For example, if that module was working in jp4.5, then flash it with pure jp4.5 BSP again.

Pure BSP means no customization.

As part of the test I mounted a known good working module with the carrier board and successfully flashed it, configured it, and logged in.

Second attempt with the problem module -Could not detect correct NVIDIA device connected to USB.

Exported the logs and attached them.

SDKM_logs_2023-07-05_14-20-12.zip (556 KB)

So are you using devkit or you are still using your coard? And which jetpack release are you using?

So what jetpack release was there for your previous customization version?

Per the request I used the reference card from the devkit. We used JetPack 4.3. No, we cannot upgrade due to multiple dependencies. We used JetPack 4.3 with ConnectTechs BSP for that version of JetPack.

Hi,

I am not sure about the situation. Does the log you provided to me really have the log that hit error?

I opened your zip file, checked the log in jp4.3 folder. Open the log of jp4.3, and it turns out a successful reflash log.

Please be aware that my previous comment didn’t mean “you must flash with jp4.5”. I am just talking about you just use same BSP to flash first.

Also, I don’t really care about whether you can upgrade or not. Since you are doing test on devkit, all you need to do is flash with any kind of jetpack version. This is just debug purpose. Do this first.