Jetson Nano Booting Issue: Stops at EMC Training

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

Hello,

Initially healthy Jetson Nano that has been operating without any issues, after next boot
Jetson Nano stopping to boot at process and stays there without finishing :
EMC Training (SRC-freq: 204000; DST-freq: 1600000)

Fan is on and at 50% Duty Cycle. Development Carrier Board is healthy, everything regarding setup is tested and verified to be healthy with other operating Jetsons.

It is not known but is possible that short has happened on the SoC which could affect on external memory controller where it stops the process.

Tests are preformed:

  1. Resetting manually from SYS_RESET == Will not boot to kernel
  2. Force recovery mode == Will enter to recovery mode
  3. Serial console on Normal boot == Processes until EMC Training (Then stops)
  4. Jetson image cloned to investigate == Cloning successfully completed; image is not flashed on other devices to be tried out
  5. Reflashed Jetson Nano == Reflashing successfully completed; Does not boot, the identical issue continues, stops at EMC Training

Serial console dump:

[0000.231] [L4T TegraBoot] (version 00.00.2018.01-l4t-c536fca9)

[0000.236] Processing in cold boot mode Bootloader 2

[0000.241] A02 Bootrom Patch rev = 1023

[0000.244] Power-up reason: pmc por

[0000.247] No Battery Present

[0000.250] pmic max77620 reset reason

[0000.253] pmic max77620 NVERC : 0x40

[0000.257] RamCode = 0

[0000.259] Platform has DDR4 type RAM

[0000.263] max77620 disabling SD1 Remote Sense

[0000.267] Setting DDR voltage to 1125mv

[0000.271] Serial Number of Pmic Max77663: 0xd06a8

[0000.278] Entering ramdump check

[0000.281] Get RamDumpCarveOut = 0x0

[0000.284] RamDumpCarveOut=0x0, RamDumperFlag=0xe59ff3f8

[0000.290] Last reboot was clean, booting normally!

[0000.294] Sdram initialization is successful

[0000.298] SecureOs Carveout Base=0x00000000ff800000 Size=0x00800000

[0000.304] Lp0 Carveout Base=0x00000000ff780000 Size=0x00001000

[0000.310] BpmpFw Carveout Base=0x00000000ff700000 Size=0x00080000

[0000.316] GSC1 Carveout Base=0x00000000ff600000 Size=0x00100000

[0000.322] GSC2 Carveout Base=0x00000000ff500000 Size=0x00100000

[0000.328] GSC4 Carveout Base=0x00000000ff400000 Size=0x00100000

[0000.334] GSC5 Carveout Base=0x00000000ff300000 Size=0x00100000

[0000.339] GSC3 Carveout Base=0x000000017f300000 Size=0x00d00000

[0000.356] RamDump Carveout Base=0x00000000ff280000 Size=0x00080000

[0000.362] Platform-DebugCarveout: 0

[0000.365] Nck Carveout Base=0x00000000ff080000 Size=0x00200000

[0000.371] Non secure mode, and RB not enabled.

[0000.387] Csd NumOfBlocks=0

[0000.625] *** Booting BFS0.

[0000.627] Read PT from (0:3)

[0000.632] Using BFS PT to query partitions

[0000.636] PT: Partition LNX NOT found !

[0001.302] *** Booting KFS0.

[0001.305] Read GPT from (0:3)

[0001.309] Using GPT Primary to query partitions

[0001.313] Loading Tboot-CPU binary

[0001.320] Verifying TBC in OdmNonSecureSBK mode

[0001.330] Bootloader load address is 0xa0000000, entry address is 0xa0000258

[0001.337] Bootloader downloaded successfully.

[0001.341] Downloaded Tboot-CPU binary to 0xa0000258

[0001.346] MAX77620_GPIO5 configured

[0001.350] CPU power rail is up

[0001.353] CPU clock enabled

[0001.356] Performing RAM repair

[0001.359] Updating A64 Warmreset Address to 0xa00002e9

[0001.365] Loading NvTbootBootloaderDTB

[0001.382] Verifying NvTbootBootloaderDTB in OdmNonSecureSBK mode

[0001.455] Bootloader DTB Load Address: 0x83000000

[0001.459] Loading NvTbootKernelDTB

[0001.476] Verifying NvTbootKernelDTB in OdmNonSecureSBK mode

[0001.549] Kernel DTB Load Address: 0x83100000

[0001.555] Loading cboot binary

[0001.566] Verifying EBT in OdmNonSecureSBK mode

[0001.607] Bootloader load address is 0x92c00000, entry address is 0x92c00258

[0001.614] Bootloader downloaded successfully.

[0001.619] PT: Partition NCT NOT found !

[0001.622] Warning: Find Partition via PT Failed

[0001.627] Next binary entry address: 0x92c00258

[0001.631] BoardId: 3448

[0001.636] Overriding pmu board id with proc board id

[0001.641] Display board id is not available

[0001.652] Verifying SC7EntryFw in OdmNonSecureSBK mode

[0001.709] /bpmp deleted

[0001.711] SC7EntryFw header found loaded at 0xff700000

[0001.907] OVR2 PMIC

[0001.908] Bpmp FW successfully loaded

[0001.913] WB0 init successfully at 0xff780000

[0001.917] Set NvDecSticky Bits

[0001.921] GSC2 address ff53fffc value c0edbbcc

[0001.927] GSC MC Settings done

[0001.931] TOS Image length 53680

[0001.934] Monitor size 53680

[0001.937] OS size 0

[0001.942] Secure Os AES-CMAC Verification Success!

[0001.946] TOS image cipher info: plaintext

[0001.950] Loading and Validation of Secure OS Successful

[0001.966] SC7 Entry Firmware - 0xff700000, 0x4000

[0001.971] NvTbootPackSdramParams: start.

[0001.976] NvTbootPackSdramParams: done.

[0001.980] Tegraboot started after 160404 us

[0001.984] Basic modules init took 1524702 us

[0001.988] Sec Bootdevice Read Time = 672 ms, Read Size = 14459 KB

[0001.994] Sec Bootdevice Write Time = 0 ms, Write Size = 0 KB

[0001.999] Next stage binary read took 7387 us

[0002.003] Carveout took -19290 us

[0002.007] CPU initialization took 354452 us

[0002.011] Total time taken by TegraBoot 1867251 us

[0002.015] Starting CPU & Halting co-processor

64NOTICE: BL31: v1.3(release):de895fd9e

NOTICE: BL31: Built : 20:52:47, Oct 27 2020

ERROR: Error initializing runtime service trusty_fast

[0002.138] RamCode = 0

[0002.142] LPDDR4 Training: Read DT: Number of tables = 2

[0002.147] EMC Training (SRC-freq: 204000; DST-freq: 1600000)

Best Regards,
M

3

hello gerhardmalberg.gm,

may I know which JetPack release you’re using.
in addition, what’s the modification you had done, when you mentioned re-flash not works, are you re-flash the native JetPack release image or your customize one?

4

Initially, I am having a custom image (Cannot provide it, sorry) which is operating on all Jetsons I have both, eMMC and SD card versions.
They are operating well without any issues.

Issue with under investigation Jetson started without making any modification at any point, it simply didn’t boot after shutting it down, moving it to another place and tried to boot again.

It is possible that a short circuit has happened on the SoC because of the setup after moving, but cause for that should be confirmed

Reflashing was done on two Jetson, the one under investigation and another one that was expected to be healthy. For the reflashing, native current latest JetPack 4.5.1 image was used on both. Both reflashed successfully.
After that,
Jetson that expected to be healthy DID booted with native image.
Jetson that was under investigation, did NOT booted with native image and continued with the same issue stopping the boot at EMC Training.

5

hello gerhardmalberg.gm,

it should normally show the status immediately,
what happened if you put the device idle and waits for EMC training results?

6

When accessing Jetson using a serial console on Normal boot (Cold boot) idle state nothing else connected to any port, then the boot stops at:

[0002.147] EMC Training (SRC-freq: 204000; DST-freq: 1600000)

And never finishes this step. Gives no status.

Can I have the information about the External Memory Controller (EMC) module location on the SoC PCB of Jetson Nano (eMMC version) so that I can check more specifically for possible mechanical/electrical damages?

8

What’s the failure rate?
Is it on devkit or custom design?
If it is devkit and there is no change on board, it looks like device issue and need RMA.
If custom design, suggest to do cross check with devkit carrier board to make sure it is module issue or custom carrier issue first.

9

It has been verified that custom carrier is healthy with other Jetson SoCs and also by placing the Jetson under investigation to Nvidia devkit carrier board. The same issue continues.

10

So it looks like a module issue, you can run RMA for it.

12

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.