Failed to boot after moving to a stand-alone environment

Hello everyone,
I have the AGX Xavier Jetson machine.
I installed on it via the Nvidia SDK Manager the Jetpack 4.4, l4t-32.4.3, which was successfully installed and I managed to use it for couple hours.
The kernel version is 4.9.140-tegra.

I had to move the Jetson to a stand-alone environment, and after that, the Jetson does not boot anymore.
It just shows the same screen while the screen always flashes. You can see the screen below:

If this information helps: the only cables that where connected to the xavier while turning it on were the power supply, HDMI, keyboad and a mouse (using a USB hub)

What does “standalone environment” mean? Did you take a Xavier AGX module off the devkit and plug it into your own carrier board via the 10,000 pin connector? If that’s the case, this kind of issue is usually caused by some driver starting up who’s hardware is no longer present (that was ON the dev kit). The solution is generally to go through the device tree and disabling different drivers for stuff you lost. Start with the sdhc interface - that’s usually an issue if it’s missing.

That ALL you see on your display? Looks like it got pretty far, but I don’t see the bootloader output.

If this is all you’re seeing you’re likely to need the microUSB to get the full Linux startup.

I also am unsure of what the standalone environment means. However, regardless of environment, is it correct to say that a different monitor is connected via HDMI and that in no case are any HDMI adapters used?

The reason I mention this is that HDMI is hot plug, and the monitor’s configuration is read via i2c during certain events. The video driver depends on this information (the EDID information is what the i2c sends). If the monitor provides a different response versus the one originally used, then it would be quite reasonable that graphical mode would fail even if the hardware is otherwise perfectly functioning.

Well first of all, standalone environment means that I had to move the Jetson to different network which isolated from the internet. This network’s router does not have DHCP which means I can’t set the IP of the Jetson (I don’t know how much this information helps, but I’m saying it anyway).

I’m using the full devkit and didn’t take the module off.
It is true that a different monitor than the one I used to flash and setup the devkit. The monitor is connected to the Jetson with a HDMI cable with no adapters used.

I’m also same facing issue after flashing the AGX Xavier from SDK Manager command line. In my case, Xavier is in same network and used same monitor after flashing it. I’m posting in a thread, but no response yet. Package dependencies error when tried to install DeepStream via SDK Manager on AGX Xavier - #3 by archana

If you can, plug an microUSB cable into the Xavier and connect that to a terminal program - teraterm, minicom - whatevery you have and try to capture the bootup output and post it.

There is also a UART1 pin (tx, rx) on the RasPi 80 pin connector that will come up with a login prompt that’s useful. You have to use a TTL-USB adapter - $3 on Amazon.

Post you bootup log- this seems diagnosable.

And you rebuilt the kernel from source, or just flashed with the SDK Manager?

Boot without networking probably results in waiting for some networking to time out and likely slows boot. Actual boot should still work even though networking would not function. What @archana mentions is what I’d also recommend, using serial console. The micro-USB connector will be in device mode if you use this with a micro-B USB cable. One of the devices made available on that connector is a serial UART, and you can run a serial console program with that to make edits and changes even though other access might be missing.

I flashed the Xavier from SDK Manager command line option.

And this is the only solution? Just use the serial console on another machine?
Is there an option to fix this situation so I can use it’s GUI?
Or do I have to reinstall Jetpack 4.4 with the flash.sh offline?

If the monitor is the issue, then flashing won’t do anything to help. It is difficult to sufficiently emphasize the fact that the video driver allows configuration only from the monitor’s self-description (known as the “EDID”). Monitors self-describe in HDMI and DisplayPort, and is why you don’t need a “driver disk” for every single monitor like in the ancient days with VGA connectors.

You cannot simply edit the “/etc/X11/xorg.conf” with manual modes (which is what the old VGA driver disk did).

The part which is particularly hard to deal with is that on a PC the drivers allow creating various custom modes if the monitor specs are in range, but in the embedded drivers, there is a set of known modes (rather extensive), but you cannot create or use “extension” modes. Only the modes directly preprogrammed are allowed. Is your new monitor within that set of modes? That’s what serial console is for, to work on finding answers about what is wrong when networking and GUI are not available.

Serial console is a direct login and works even when much of the system is dead or burning. Want to see logs when the system is failing? Use serial console. Want to edit something because it is hard to get the system fully operational without that edit? Use serial console. Want to set something up without a keyboard/monitor on the Jetson? Use serial console. This is why I mention serial console: So you can find out what what is going on and possibly fix an issue without reflashing.

So far as I know this system is not networked, and the monitor/GUI is not working. The system used to work, but on a different monitor. The two ways of moving forward would be to either use serial console, or to put the old monitor/keyboard back on.

If the monitor specs are the reason for not working, then the system is not really “broken”, and flashing is not a fix.

So what are you saying is that probably the problem is the monitor I used is not compatible with the Jetson systems.
Maybe I get it wrong but if this is the case so I was supposed to not see any output in the monitor?

Correct. If the monitor does not directly have the EDID modes supported without extensions, then the GPU uses a fallback setting. If the fallback is not useful to the monitor, then it just remains black.

Now I installed minicom on my Ubuntu desktop, then tried below command and rebooted Xavier, but can’t see any output on screen:

    archana@archana-H310M-DS2:~$ lsusb
    Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
    Bus 001 Device 005: ID 046d:0825 Logitech, Inc. Webcam C270
    Bus 001 Device 004: ID 413c:2113 Dell Computer Corp. 
    Bus 001 Device 003: ID 0000:0538  
    Bus 001 Device 011: ID 0403:6011 Future Technology Devices International, Ltd FT4232H Quad HS USB-UART/FIFO IC
    Bus 001 Device 002: ID 148f:7601 Ralink Technology, Corp. MT7601U Wireless Adapter
    Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Output for ‘dmesg | grep tty’:

[    0.070721] printk: console [tty0] enabled
[    3.944941] 00:02: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 2524.256654] usb 1-3: FTDI USB Serial Device converter now attached to ttyUSB0
[ 2524.256873] usb 1-3: FTDI USB Serial Device converter now attached to ttyUSB1
[ 2524.257093] usb 1-3: FTDI USB Serial Device converter now attached to ttyUSB2
[ 2524.257347] usb 1-3: FTDI USB Serial Device converter now attached to ttyUSB3

I never used minicom any similar apps before, but tried few commands:

archana@archana-H310M-DS2:~$ sudo minicom /dev/ttyUSB0

Welcome to minicom 2.7.1

OPTIONS: I18n 
Compiled on Aug 13 2017, 15:25:34.
Port /dev/tty8, 11:46:48

Press CTRL-A Z for help on special keys

I tried with all ttyUSB0, ttyUSB1, ttyUSB2, ttyUSB3 in minicom and getting only few line as above. Let me know if I need to try any other steps with minicom.

Have you set baud to 115200 and turned off flow control. Likely USB3 is the console - it should show the entire bootup sequence.

CTL-A S to get to setup mode.
You maybe need to chmod 777 /dev/ttyUSBx once per port as well.

Post you capture log.
Thanks
LArry

Basically, what @larry.ciummo just said. I will add though that “minicom” has a long history from the days when people used slow modems over the telephone. Much of minicom is to support special commands to “ppp” to tell the modem how to dial and connect/authenticate with an ISP. There is a lot there you don’t need, but minicom is a well known and reliable tool.

I recommend simplifying and using gtkterm (“sudo apt-get install gtkterm”) from the host PC, instead of minicom (unless you are already familiar with minicom and have it working). Naming the right tty is key, but in gtkterm, if you’ve named the correct /dev/tty...something..., then it looks like this (I’m using “/dev/ttyUSB0” only as an example, you’d have to adjust and find the right one):
gtkterm -b 8 -t 1 -s 115200 -p /dev/ttyUSB0
(this connects with the requires speed 11500, 8 bits, no stop bit, and one parity bit)

If it works, then you can reboot and see content from much earlier in boot. Tools like gtkterm (and minicom) have an option to log the session. Once you have this working, and the Jetson is powered off, then you can clear the screen and start logging, followed by booting, and once done booting, turn off logging. You’ll have a nice log file to attach to the forum.

Good idea from @linundev. I must be getting old - my next term program suggestion was going to be “tip”.

For me gtkterm or minicom doesn’t show any output. I used picocom suggested here. So I posted the logs there, not sure if I follow the thread there only. Because now, it may repeat same things.

Yes, picocom is also a good choice.