/dev/ttySn: n>3; minicom; debugging serial peripheral connected to PCI Express

Hello, John here.

My COTS PCIe card’s four RS232 ports don’t communicate in minicom, nor anywhere else I’ve tried.

Initially I had issues rebuilding the kernel to support more than 4 ttyS (ttyS0 appears to be an onboard uart so my PCIe board will consume ttyS1->ttyS4). I’ve increased SERIAL_8250_NR_UARTS in config and am thus able to MAKEDEV ttyS4.

I then issued four setserial /dev/ttyS1 port 0x1000 irq 388 baud_base 921600 commands (changing “ttyS1” and “0x1000”, as appropriate). irq “388” is as determined by lspci -vv, but lspci -bvv reports a different number, “132”, and I’m not sure which I should use.

If I use setserial’s “auto_irq” doesn’t change whatever the current IRQ is configured to, whether it is 388 or 132 … or 133.

If I use setserial’s “autoconfig” it doesn’t change anything, either.

In all cases the uart lists “unknown”, while I expected “ST16550” or “16550A” or similar.

These UARTs are IO devices (0x1000, 0x1008, 0x1010, etc) on a PCI (express) PNP card and the Linux driver file /drivers/tty/serial/8250/8250_pci.c has the VENDEV etc., but the TX2 doesn’t support ACPI so it doesn’t support 8250_PNP, so that file doesn’t get used.

Anyway, if /dev/ttyS1 has been set up with “sudo setserial /dev/ttyS1 port 0x1000 baud_base 921600 irq 388 uart 16550A” then I issue minicom -D /dev/ttyS1 it freezes.

What’s the correct approach to diagnosing this?

Connect a port’s TX to RX and CTS to RTS…use it in loopback. Use your serial console application pointed at this and see if it echos back. If no echo, then that is one clue…if echo is garbage, then that is a different clue. Normally there is always a question of whether both ends of the communications are matched in settings…but a port will always agree with itself.

One of the problems with older serial equipment is that it isn’t truly plug-n-play. The hardware itself does not report its settings to the driver. All settings for set and query are with the driver. If the driver is happy or thinks that is the setting, then it won’t have any way to know the hardware can’t do that. Bliss is ignorance :P

Some serial programs you might use during your testing (the picocom one is from a PC talking to a serial console without flow control…I usually edit the “.gtkterm” file directly and use gtkTerm):

        picocom -b 115200 --databits 8 --parity n --stop 1 --flow n /dev/whatever/it/is

Have you updated the device tree to match your changed mknod drivers? The Jetson TX2 is not a native plug-and-play system, so devices often need to be declared in the device tree, and it’s important to not overlap. I’d expect the Jetson to want ttyS0…3, and you should make yours be ttyS4…7.

Also, 8250 and derived versions is a terrible hardware interface for modern CPUs. Does your PCI express card not have any better interface and drivers, perhaps using native DMA?


I agree, always a good idea. I get no echo under some test conditions, and kernel faults under others.

This device, and its drivers, are baked into Linux, but as PCI/PNP using entries in 8250_pci.c — which doesn’t appear to get used in the TX2 distro.

Thanks for the list of serial-related utilities in Linux: I was only familiar with setserial, minicom, and PuTTy, so I’m glad for the help.


No, and I don’t know how. In fact, the only device tree I’m familiar with is the FDT in VxWorks.

The Jetson only had ttyS0 reserved; my card’s ports “already” shows up as ttyS1-ttyS3 “out of the box”, but they don’t work, are in a strange order vs the register map’s expected order, and ttyS4 is missing (the “nr_uarts” issue).

The chip supports both the 16550A “simple” mode using I/O registers and very enhanced operations using MEM registers, but the MEM interface is complex and I don’t have a driver I can base a VxWorks port on. The chip manufacturer doesn’t support the more complex interface in Linux and doesn’t share the Windows source without an NDA [which I have, and which precludes porting].

Although the UART I/O register interface is a terrible interface, the most common baud rate in the industrial market is still 9600 — so it is “plenty”.

Although I haven’t looked for other UARTs, the TX2 does have at least four (only four general use UARTs that I know of) UARTs. The nomenclature of how those UARTs appear in “/dev/” depends on which driver is running (or available for) the UART. You will find “/dev/ttyS0” uses the 16550A non-DMA driver. The reason this non-DMA driver was used on ttyS0 is because the serial console in U-Boot does not understand DMA mode. Had U-Boot understood DMA and the NVIDIA driver had been used instead it would have been “/dev/ttyTHS0” (“S” implies Serial, “THS” implies Tegra High Speed serial). Each of those UARTs can be set to different emulations and used by one of those drivers at a time. Switching drivers when going between U-Boot and Linux would have been a loss of continuity of service and would risk losing serial console messages (kind of pointless to mention since U-Boot doesn’t understand DMA anyway…it’s just a point of illustrating driver ownership).

The rest of the UARTs would normally be accessed only after the Linux kernel is running and occupy “ttyTHS1” through “ttyTHS3”. I believe that you will also see “ttyS1” through “ttyS3”, but these are in fact the same three UARTs and not six different hardware devices. Even if you did not see “ttyS1” through “ttyS3” used, then those slots are probably still used by the integrated UARTs and moving to “ttyS4” through “ttyS7” may still be required.

In the device tree take note that the “compatible” for the integrated UARTs will have multiple drivers listed as possible. Example:

serial@3110000 {
                reg = <0x0 0x3110000 0x0 0x40>;
                dmas = <0x19 0x9 0x19 0x9>;
                interrupts = <0x0 0x71 0x4>;
                reg-shift = <0x2>;
                <b>compatible = "nvidia,tegra186-hsuart"</b>;

Either of the “compatible” drivers can be used. Both device special files can be generated by a single UART. Only one driver can be used at a time (actively using two drivers on a single UART is asking for trouble).


Good to know! If the fact that my PCIe card’s UART I/O BAR is given address 0x1000, and that ttyS1, ttyS2, and ttyS3 respond to setserial -vvvv as being configured to 0x1008, 0x1038, and 0x1010 from the card’s BAR, is merely a flaw in tegra-linux and I should remove those three ttySn and MAKEDEV 4, 5, 6, and 7 instead to fix it I’ll be happy.

But: How can I cause a customer’s TX2, up and running, to correctly detect and install my card’s UARTs as ttyS4-7 once the card is installed? (assuming they power it down, install the card, and power it back up, of course, not looking to hotplug!)

Is it a change needed in the tegra linux configuration, to mark ttyS0-3 as already used (somehow. I assume it is possible.)

If your card depends on drivers that map the inode ID to memory addresses.
AND the Jetson already uses ttyS0…3 (which it does)
AND the ID of your card is hard-coded to the ttyS0…3 range.
AND the 8250 ttyS driver hard-codes the number to the IO range.
THEN you need to build a parallel driver that uses a different name (ttyPCIe0…3 for example)

Driver can be loaded as modules with “modprobe” and can install whatever devices they want when they do so.
Typically, this is set up through hot-plugging with udev rules, and sometimes with systemd rules (“unit files”)
If you just want something manual, a script that does the appropriate “mknod” calls might also work.

Also, I’m not sure which driver on the Jetson generates ttyS0…3. It’s probably not the 8250 driver. Thus, the “mknod” to create the ttySx devices is unlikely to actually work right.
You need some other way to tie a TTY name to the actual driver code/module you want to use.

I have never adjusted this, but you might investigate these kernel features (this is an excerpt of “zcat /proc/config.gz”):


…perhaps you can simply up this to 8 (or 5?), depending on whether your device has those names hard coded.

@linuxdev: I definitely adjusted those! My company discovered those kernel options back when the default changed as we make more than three-dozen serial boards with as many as 8 ports.

(I had much difficulty figuring out how to get the change to STICK, but eventually found instructions on how to update the default config instead of changing it after every time I ran make mrproper) (why can’t I use make clean instead, isn’t that what it is for? The makefile errors out. grr.)

When I set nr_uarts high enough I can successfully

MAKEDEV /dev/ttyS4

and I can then

setserial /dev/ttyS4 port 0x1038 baud_base 921600 irq $IRQ_GRR

but subsequent attempts to use ttyS4 cause endless kernel panics and I’m not ready to debug them. (minidump parser, I suppose.)

Instead: I just want to write a program like:

outb(0xaa, 0x1007); printf("%02X ",inb(0x1007));

just to verify I can access the scratchpad register. As it stands, I always read zero.

(I coded outb/inb using open/lseek/read/write on /dev/ports but no luck, just 0x00 reads from all addresses.)

Thank you, tremendously, for your assistance.

Typically I use “make mrproper”, but have a copy of the config saved somewhere outside of the tree (I start with a working system’s “/proc/config.gz” with the CONFIG_LOCALVERSION set). Within the build area I completely delete everything instead of relying on any kind of “clean” target (this seems to have become more of a necessity since some of the files downloaded from source_sync.sh are not in the actual kernel tree).

Have you tried the utility “devmem2” yet (“sudo apt-get install devmem2”, “man devmem2”)? Even if you don’t want to debug a kernel error it might be nice to post the error here and name the command which triggers it. The dmesg of the error should show a stack frame which would be useful.

Remember that the setserial command is telling a driver what settings to use…this doesn’t guarantee the hardware is actually there or complying with the changes.

I am curious what you see from:

sudo find /sys -type d -name '*serial*'

You’ll notice that a group of the directories found in that command lists the base address of the controller working with the serial device. I’d be interested in seeing what base addresses show up which are not from the Jetson’s integrated UARTs. For reference, here is the output from a stock R28.2 install:

# find /sys -type d -name '*serial*'