We’ve had several instances of the debug serial port ceasing to function seemingly permanently. Interestingly, when the debug port stops outputting anything it does appear to still receive - we can type commands into the terminal that the Xavier responds to. But it doesn’t send anything out (i.e. it seems like rx is working but not tx).
We do have our own custom carrier board. However, if we swap in a different xavier, the debug serial port is still functioning normally in both directions. So it seems relatively unlikely to be an issue with our carrier board.
We have our own process for updating the software running on the xavier and that process can sometimes change the installed DTB file linux uses to boot. Initially we hypothesized that one of the changes we make there caused the serial port to stop functioning but we tested this theory by taking a xavier with a functioning debug serial port and applying our changes incrementally and the serial port continued to function after all changes were applied.
So we are somewhat at a loss for why the debug serial port would stop working and not be recoverable. Has anyone experienced anything similar or have ideas on how to re-enable the debug serial port?
Update: the issues with the debug port not working with some AGX modules is also present on a development kit. In other words, I’m currently testing with two AGX modules. For one of them the debug port works properly on both the development kit and our custom board. For the other the debug port TX does not work on either the development kit or our custom board.
If the voltage of the outside UART has incorrect levels versus the carrier board, then being out of spec it might still function in some way. Working one direction, but not the other, is a common result of something like mixing 3.3V and 5V or 1.8V.
If flow control was somehow enabled on the side failing to receive, then the UART would be functioning perfectly and you’d see half of the connection cut off.
I’d suggest an oscilloscope on TX and RX, and see if there is actual traffic on the failing direction. It wouldn’t even matter if you have a logic analyzer, you’d just need to know (A) TX is really sending, and (B) the voltage level looks sane.
Thanks for your reply. I have scoped the line itself and its just low (0V). And I’m not sure the schematic of the dev kit but on our custom board there is a pull-up to 3v3 as part of a level shifter (shifting the Xavier’s 1v8 to 3v3). So for the line to be low the Xavier actually needs to be driving it low. And since that isn’t even the inactive voltage for UART my suspicion is that the pin has somehow been configured to be an output but not a UART pin.
If you scope that particular wire from power off to boot, is it ever high once power is up? Knowing if the low starts at (A) power on, versus (B) after loading the Linux kernel, would perhaps offer some insight.
Also, is this just logic level 0 voltage, or is it actually 0? Does the other module work on this exact carrier board? I ask because this would be a specific electrical fault if it is actually 0 or if the other module works on a different carrier board, but not on that board. It sounds like it is software, but it is a question in need of verification.
Also, is anything printed in logs when trying to use TX?
Finally, the device tree makes definitions of pins and routing. If you compare the device tree node for that specific GPIO, do they match between both modules? Are you using the device tree in the module’s eMMC “/boot”? Or are you using the module in a partition? Verifying that (A) the exact same device tree is loaded and used, and (B) the specific node once booted (see “/proc/device-tree/” to verify if the node is an exact match after the two units are booted) is an exact match?