How to disable IOMMU for pcie c5-rp on Jetpack6.0

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

Hi Nvidia,

Follow the modify described in How to disable IOMMU in Jetson Orin - #4 by WayneWWW . I successfully bring up the dma transfer through pcie in Jetpack5.1.1.

Now I’m going to upgrade the kernel version, so I did the same change in Jetpack6:

  1. Remove below properties in tegra234.dtsi, pcie@141a0000 {
// iommu-map = <0x0 &smmu_niso0 TEGRA234_SID_PCIE5 0x1000>;
// iommu-map-mask = <0x0>;
// dma-coherent;
  1. and in tegra234-base-overlay.dtsi, pcie@141a0000 {
// iommus = <&smmu_niso0 TEGRA234_SID_PCIE5>;
  1. Replace the dtb file and reflash the orin.

But there will be errors when transfer data through pcie-c5:

[  120.461477] pcieport 0005:00:00.0: AER: Root Port link has been reset (0)
[  120.461521] pcieport 0005:00:00.0: AER: device recovery failed
[  120.461523] pcieport 0005:00:00.0: AER: Multiple Uncorrected (Fatal) error received: 0005:00:00.0
[  120.461554] pcieport 0005:00:00.0: PCIe Bus Error: severity=Uncorrected (Fatal), type=Transaction Layer, (Receiver ID)
[  120.461555] pcieport 0005:00:00.0:   device [10de:229a] error status/mask=00040000/00400000
[  120.461557] pcieport 0005:00:00.0:    [18] MalfTLP                (First)
[  120.461559] pcieport 0005:00:00.0: AER:   TLP Header: 40000040 010002ff ff400000 68656c6c

And iommu errors as well

image
image1254×370 28.6 KB

However, pcie works fine when I do the same modify in Jetpack5.1.1.

Is there anything difference of iommu between JP6 and JP5.1.1?

5

For more proof, I believe the iommu related properties has been removed because I can’t see them in the iommu device:

image
image1253×308 12.6 KB

and in device-tree:

image
image1182×162 7.99 KB

6

please attach full error log but not screenshot…

7

Hi Wayne,

Here is the full dmesg log and lspci log.

dmesg.txt (78.6 KB)
lspci.txt (26.1 KB)

Please don’t mind the print like “phys_to_dma_direct” which are for my debug purpose.

And the device tree re-compile from the dtb file in /boot/dtb/
dtb.txt (314.0 KB)

8

Hi Wayne,

Any update? I haven’t found out the root cause yet. We are now facing the risk to degrade the jetpack version now, so I’m looking forward to your reply.

Thanks!

10

i don’t think it’s the iommu’s problem, cause our driver don’t even need to disable the pcie’s iommu ,still report a AER in the new jetpack 6.0 . so i think either it’s the ports driver’s problem or the dt’s problem . i notice the new dt is simplifed a lot from the old one , i will keep tring to change the dt to have a test.

11

My driver works fine on jp5.1.1 with iommu disabled.
On jp6.0, I disable the iommu by the same way as I do in jp5.1.1, but my driver still reports iommu error.
I’m using exactly the same orin and same driver, the only thing different is the jetpack version.

I notice the huge amount of significant difference in pcie kernel driver, memory driver and device tree between jp6 and 5.1.1, but they seem too complicated for recognizing.

13

Hi,

We are checking this internally. will give feedback later.

15

any update?

16

SMMU bypass is not allowed due to security reasons. Upstream added this restriction by setting CONFIG_ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT=Y default from latest kernel.

If you need this, please set above to =N.

18

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.