PCIe issue on igb device

Hello,

It has one PCIe issue when I use Ubuntu 16.04 ( rootfs and kernel are 64-bits)

[ 5.352189] vgaarb: this pci device is not a vga device
[ 5.409851] vgaarb: this pci device is not a vga device
[ 7.192330] smmu_dump_pagetable(): fault_address=0x00000000fdd06000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[ 7.203709] mc-err: (0) csr_afir: EMEM decode error on PDE or PTE entry
[ 7.210344] mc-err: status = 0x6000000e; addr = 0xfdd06000
[ 7.215993] mc-err: secure: no, access-type: read, SMMU fault: nr-nw-s
[ 9.291371] smmu_dump_pagetable(): fault_address=0x00000000fdd0e000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[ 9.302763] mc-err: (0) csr_afir: EMEM decode error on PDE or PTE entry
[ 9.309417] mc-err: status = 0x6000000e; addr = 0xfdd0e000
[ 9.315084] mc-err: secure: no, access-type: read, SMMU fault: nr-nw-s
[ 11.031861] smmu_dump_pagetable(): fault_address=0x0000000000000000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[ 11.043214] mc-err: (0) csw_afiw: EMEM decode error on PDE or PTE entry
[ 11.049933] mc-err: status = 0x60010031; addr = 0x00000000
[ 11.055591] mc-err: secure: no, access-type: write, SMMU fault: nr-nw-s
[ 11.076823] smmu_dump_pagetable(): fault_address=0x00000000fdd0c000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[ 11.088160] mc-err: (0) csr_afir: EMEM decode error on PDE or PTE entry
[ 11.094773] mc-err: status = 0x6000000e; addr = 0xfdd0c000
[ 11.100456] mc-err: secure: no, access-type: read, SMMU fault: nr-nw-s
[ 11.242905] smmu_dump_pagetable(): fault_address=0x0000000000000000 pa=0xffffffffffffffff bytes=ffffffffffffffff #pte=0 in L2
[ 11.254301] mc-err: (0) csw_afiw: EMEM decode error on PDE or PTE entry
[ 11.260945] mc-err: status = 0x60010031; addr = 0x00000000
[ 11.266604] mc-err: secure: no, access-type: write, SMMU fault: nr-nw-s
[ 11.278581] mc-err: Too many MC errors; throttling prints
[ 13.047509] igb 0000:03:00.0: Detected Tx Unit Hang
[ 13.047509] Tx Queue <2>
[ 13.047509] TDH <1>
[ 13.047509] TDT <1>
[ 13.047509] next_to_use <1>
[ 13.047509] next_to_clean <0>
[ 13.047509] buffer_info[next_to_clean]
[ 13.047509] time_stamp
[ 13.047509] next_to_watch
[ 13.047509] jiffies
[ 13.047509] desc.status <168000>
[ 15.046886] igb 0000:03:00.0: Detected Tx Unit Hang
[ 15.046886] Tx Queue <2>
[ 15.046886] TDH <1>
[ 15.046886] TDT <1>
[ 15.046886] next_to_use <1>
[ 15.046886] next_to_clean <0>
[ 15.046886] buffer_info[next_to_clean]
[ 15.046886] time_stamp
[ 15.046886] next_to_watch
[ 15.046886] jiffies
[ 15.046886] desc.status <168000>
[ 17.046895] igb 0000:03:00.0: Detected Tx Unit Hang
[ 17.046895] Tx Queue <2>
[ 17.046895] TDH <1>
[ 17.046895] TDT <1>
[ 17.046895] next_to_use <1>
[ 17.046895] next_to_clean <0>
[ 17.046895] buffer_info[next_to_clean]
[ 17.046895] time_stamp
[ 17.046895] next_to_watch
[ 17.046895] jiffies
[ 17.046895] desc.status <168000>
[ 19.046904] igb 0000:03:00.0: Detected Tx Unit Hang
[ 19.046904] Tx Queue <2>
[ 19.046904] TDH <1>
[ 19.046904] TDT <1>
[ 19.046904] next_to_use <1>
[ 19.046904] next_to_clean <0>
[ 19.046904] buffer_info[next_to_clean]
[ 19.046904] time_stamp
[ 19.046904] next_to_watch
[ 19.046904] jiffies
[ 19.046904] desc.status <168000>
[ 21.046902] igb 0000:03:00.0: Detected Tx Unit Hang
[ 21.046902] Tx Queue <2>
[ 21.046902] TDH <1>
[ 21.046902] TDT <1>
[ 21.046902] next_to_use <1>
[ 21.046902] next_to_clean <0>
[ 21.046902] buffer_info[next_to_clean]
[ 21.046902] time_stamp
[ 21.046902] next_to_watch
[ 21.046902] jiffies
[ 21.046902] desc.status <168000>
[ 21.092379] igb 0000:03:00.0 eth1: Reset adapter

and here is the lspci & ifconfig:

ubuntu@tegra-ubuntu:~ lspci -nk 00:01.0 0604: 10de:0fae (rev a1) Kernel driver in use: pcieport 01:00.0 0604: 12d8:2303 (rev 05) Kernel driver in use: pcieport 02:01.0 0604: 12d8:2303 (rev 05) Kernel driver in use: pcieport 02:02.0 0604: 12d8:2303 (rev 05) Kernel driver in use: pcieport 03:00.0 0200: 8086:1533 (rev 03) Kernel driver in use: igb Kernel modules: igb 04:00.0 0200: 8086:1533 (rev 03) Kernel driver in use: igb Kernel modules: igb ubuntu@tegra-ubuntu:~ ifconfig
enx00044b634602 Link encap:Ethernet HWaddr 00:04:4b:63:46:02
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

eth1 Link encap:Ethernet HWaddr 00:00:00:00:00:20
inet6 addr: fe80::200:ff:fe00:20/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Memory:13000000-13100000

eth2 Link encap:Ethernet HWaddr 00:00:00:00:00:21
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Memory:13200000-13300000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:244 errors:0 dropped:0 overruns:0 frame:0
TX packets:244 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:18000 (18.0 KB) TX bytes:18000 (18.0 KB)

wlan0 Link encap:Ethernet HWaddr 00:04:4b:63:46:00
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

The NIC does not work.
But if I change the rootfs to older version 14.04 (32-bits) and use the same kernel,
it would be work.

Does anyone know how to solve it?
Thanks a lot.

Hi,

See https://devtalk.nvidia.com/default/topic/967515/l4t-r24-2-pcie-iommu-not-working-when-switches-bridges-are-present/

Hi IsaHung,
Is your Ethernet card sitting behind a PCIe bridge? If yes, apply the patch present in link of comment #2. If not, can you describe your system a bit?

Hello iemdey and vidyas,

After I apply the patch,
the issue is fixed.
Thanks you very much!