Tegra 30 PCIe

We are using an apalis T30 module from Toradex (www.toradex.com). We are trying to integrate support for PCIe uart XR17V354 and found having issues in receiving characters. On examining the issue in detail, we found that the RX FIFO can only be accessed with readl . If we use readb, the contents are lost.

However, the same work without any issues in i.MX6 platforms.

The Kernel in use is a 3.1.10. Any Idea how this can be solved ?

Any ideas how this can be fixed ? Please let me know if the description of issue needs more details.

Regards,
Don.

I can’t answer the question, but how is that memory location listed in “/proc/iomem” (it would be of interest to know how the kernel thinks of that address range)?

Hi

Thanks.

Please see the output of 1. dmesg when loading the driver, 2. lspci and 3. cat /proc/iomem

[ 41.579330] Exar PCIe (XR17V35x) serial driver Revision: 2.2
[ 41.585489] XR17V35x : MSI interrupts enabled
[ 41.589955] 0000:04:00.0: ttyXR0 at MMIO 0x10200000 (irq = 486) is a XR17v35x
[ 41.630573] init_one_xrpciserialcard line:0
[ 41.634833] 0000:04:00.0: ttyXR1 at MMIO 0x10200400 (irq = 486) is a XR17v35x
[ 41.670549] init_one_xrpciserialcard line:1
[ 41.674808] 0000:04:00.0: ttyXR2 at MMIO 0x10200800 (irq = 486) is a XR17v35x
[ 41.710551] init_one_xrpciserialcard line:2
[ 41.714809] 0000:04:00.0: ttyXR3 at MMIO 0x10200c00 (irq = 486) is a XR17v35x
[ 41.750550] init_one_xrpciserialcard line:3
[ 132.304390]

root@apalis-t30:~# lspci -xvd 13A8:*
04:00.0 Serial controller: Exar Corp. Device 0354 (rev 03) (prog-if 02 [16550])
Flags: bus master, fast devsel, latency 0, IRQ 486
Memory at 10200000 (32-bit, non-prefetchable)
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [78] Power Management version 3
Capabilities: [80] Express Endpoint, MSI 01
Kernel driver in use: xrserial
Kernel modules: 8250_pci
00: a8 13 54 03 46 05 10 00 03 02 00 07 08 00 00 00
10: 00 00 20 10 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 82 01 00 00

root@apalis-t30:~#

root@apalis-t30:~# cat /proc/iomem
03000000-030fffff : PCI IO
10000000-1fffffff : PCIe MEM Space
10000000-100fffff : PCI Bus 0000:07
10000000-1001ffff : 0000:07:00.0
10000000-1001ffff : igb
10020000-10023fff : 0000:07:00.0
10020000-10023fff : igb
10200000-10ffffff : PCI Bus 0000:01
10200000-10ffffff : PCI Bus 0000:02
10200000-102fffff : PCI Bus 0000:04
10200000-10203fff : 0000:04:00.0
10400000-10ffffff : PCI Bus 0000:05
10400000-10403fff : 0000:05:00.0
10400000-10403fff : igb
10800000-10ffffff : 0000:05:00.0
10800000-10ffffff : igb
20000000-3fffffff : PCIe PREFETCH MEM Space
50000000-50023fff : host1x
50000000-50023fff : host1x
54040000-5407ffff : regs
54040000-5407ffff : mpe

console_log.txt (2.13 KB)

I should add that I was wanting to compare the actual address you used with readl/readb. What is the actual address in both cases so that the iomem data can be compared?

Hi Guys,

We are using the same XR17V354 UART and dealing with the same situation. It works on Intel based platform but nothing appears on serial when we mounting this module (mini PCIe) on Jetson TX2.
Do you managed to solved this?

You should probably ask in the TX2 forum since it isn’t legacy hardware:
https://devtalk.nvidia.com/default/board/188/jetson-tx2/

Typically a desktop system has far more drivers present than actually used. If this isn’t an FTDI chipset UART, then chances are it simply needs a driver added. I am not familiar with the XR17V354 so I don’t know what driver it uses.

Hello

Sorry for replying so late, the priority was little off with this one.

We did some more tests on this, here are some observations.

Here we made a skeleton driver and a matching application. With this combo, we can read/write any register and fill/dump FIFOs inside exar. And during trials on T30, we could see that byte wise fetch from exar RX FIFO fails. As a result, all functions like readb(), memcpy_fromio() etc fails when it is used to read FIFO. (By fails, we mean from a full FIFO, if we read content byte wise , one by one,
The first byte is always copied, but consecutive bytes are lost.). As a result, the only way in which we can access FIFO without data loss is by using readl(), copying 4 bytes at a time. This make sense also, as PCI probably trying to fetch four bytes at once and reporting only one byte. And since this is a FIFO, the read four bytes are popped from FIFO, but three bytes lost forever.

We did the same on an i.MX6 platform. Interestingly, this is different with i.MX6. With i.MX6, we can access FIFOs byte wise and don’t result in data Loss. So we guess this issue is either T30 PCI host hardware not supporting byte wise access, or the host side driver not supporting this access.

We have a intel i210 pcie gigabit ethernet sharing this pci line. This uses IGB driver. We did a check on IGB drivers. IGB driver works without issues on same PCI bus / host driver combo. There we could see all FIFO access is made using readl(). So we think modifying uart driver to use readl() to
fetch from FIFO could solve the issue. We have not tried it though.

Thank you DonMichael for finding time and sending us more information.
We open a channel with Exar (UART manufacturer).

Hello, I’m working with a similar Exar card and would like to hear if you resolved the driver or configuration issue that prevents the uarts from working. I have an open issue with the vendor and am actively debugging the driver in the meantime. I will try DonMichael’s suggestion tomorrow, too.

Exar has an updated driver ( https://www.maxlinear.com/document/index?id=22068&type=Software+Drivers&partnumber=XR17V354 ). Did someone try with those already ? As per their changelist, they have done the correction for these kind of issues.

From changelog : Changes from Ver 2.2 to 2.2a

  • For receiving data, replaced memcpy_fromio() routine with serial_in(). This was done because the former was dropping data on receive.

I’m running v2.4 from April 2018 right now. It appears to have the same code (I haven’t diff’ed it) tweaks related to the memcpy_fromio() in transmit_chars, but it does not work.

I’ll have more debugging info by this evening, hopefully.

Hi Guys,

Just to update you. We gave up eventually trying to resolve the issue due to the project time constrains.
I used FTDI USB-to-serial board, connected it to USB and it worked perfectly for me. Example of 232, 422, 485 modules: https://www.ftdichip.com/Products/Modules/USBRSxxx.htm

My issue seems to be more related to interrupts. When I open the port and configure its baud etc I see the port settings updating from my printk statements. Then I send characters from minicom and I see serialxr_start_tx method run which enables the interrupts in the Exar IC, but interrupts are never raised. I appear to be properly requesting an interrupt with a valid callback function and receiving one from the system with no errors…this code I did not change at all from Exar’s source.

Because of this I’m never getting to the transmit_chars function where Don’s suggestion would be applied.

The Exar card works in my Ubuntu 18.04 64bit Intel desktop. I tried compiling Exar’s source on my desktop to watch the output, but I can’t build it due to some kernel differences.

Right now I’ve switched back to the Jetson and am trying to add printk’s to 8250_pci.c (and related files) to debug the default driver which also doesn’t work right now, but I have no info about where it is failing yet.

Ok, I got the Exar driver building on my PC and I can clearly see a functioning pattern. When characters are received by the tty start_tx is called, then the IER bit of the interrupt register is set (if not already set), then an interrupt comes in, and finally transmit_chars is run.

On Jetson I see start_tx, IER enable, then nothing. However, early in the driver init I do see 3 interrupts come in, but then never again until I remove and reinsert the driver. Maybe this is is because readl is breaking the FIFO interrupt…I’ll implement Don’s idea later today and see if my interrupts start working consistently.

I am attaching two files from the time I was debugging it. Maybe it will give you some direction.

xr17v35x_aitech.c (60.7 KB)
xr17v35x.c (60.5 KB)

I attached files in previous comment

Awesome, thank you!