Hi
I’m working on transfer data between 2 Orin through PCIE switch. After using dma_map_xxx to map source address(by kmalloc) and destination address(BAR physical address), I submit the DMA transfer and there are errors like:
d@d-desktop:/sys/kernel/debug/ntb_perf/0005:01:00.1$ sudo echo 0 > run
[ 238.539299] irq 14: nobody cared (try booting with the "irqpoll" option)
[ 238.539758] handlers:
[ 238.539832] [<0000000074e49222>] tegra_mcerr_hard_irq threaded [<00000000b6412363>] tegra_mcerr_thread
[ 238.540103] Disabling IRQ #14
[ 238.559003] mc-err: (255) csw_pcie5w: EMEM address decode error
[ 238.559423] mc-err: status = 0x200100e3; hi_addr_reg = 0x00000027 addr = 0x2740080200
[ 238.559897] mc-err: secure: no, access-type: write
[ 238.646719] mc-err: (255) csw_pcie5w: EMEM address decode error
[ 238.647208] mc-err: status = 0x200100e3; hi_addr_reg = 0x00000027 addr = 0x27400b3c00
[ 238.647736] mc-err: secure: no, access-type: write
d@d-desktop:/sys/kernel/debug/ntb_perf/0005:01:00.1$ [ 238.754684] mc-err: (255) csw_pcie5w: EMEM address decode error
[ 238.754872] mc-err: status = 0x200100e3; hi_addr_reg = 0x00000027 addr = 0x27400b2e00
[ 238.755290] mc-err: secure: no, access-type: write
[ 238.858871] mc-err: (255) csw_pcie5w: EMEM address decode error
[ 238.859160] mc-err: status = 0x200100e3; hi_addr_reg = 0x00000027 addr = 0x2740080000
[ 238.859506] mc-err: secure: no, access-type: write
[ 238.963471] mc-err: Too many MC errors; throttling printsd@d-desktop:/sys/kernel/debug/ntb_perf/0005:01:00.1$
However I can see the data flow on my switch monitor.
But it seems that the data has not been carried to the other Orin. I am quite sure there is no problem with the switch because if I use memcpy_toio(instead of DMA) with exactly the same source address and destination address, it works and there is data received by the other Orin.
I prefer there are something wrong with Orin’s IOMMU which is possible that the IOMMU is blocking access by the switch-dma to host memory.
Is this because of iommu? How to fix it?
My BSP version is JetPack 5.0.1.
Thanks.