# Orin NX R36.2 cannot change IRQ smp affinity

**URL:** <https://forums.developer.nvidia.com/t/orin-nx-r36-2-cannot-change-irq-smp-affinity/280988>\
**Category:** Jetson Orin NX\
**Tags:** nvbugs, kernel\
**Created:** [February 1, 2024, 4:24pm UTC](https://forums.developer.nvidia.com/t/orin-nx-r36-2-cannot-change-irq-smp-affinity/280988 "2024-02-01T16:24:32Z")\
**Posts on this page:** 1\
**Showing post:** 7

<div class="post-metadata">

**Author:** ![thegtx25](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@thegtx25](https://forums.developer.nvidia.com/u/thegtx25)\
**Post date:** [February 2, 2024, 1:37pm UTC](https://forums.developer.nvidia.com/t/orin-nx-r36-2-cannot-change-irq-smp-affinity/280988/7 "2024-02-02T13:37:46Z")

</div>

Hi KevinFFF,

We are using a Seeedstudio A603 carrier which is similar to devkit. There is no need for specific dtbs update to have it work with R35 or R36.

I have tried putting back the msi-controller with GIC\_V2M as in R35 and attaching the C1 pcie port to it.

I have assumed that the SPIs for PCIe MSI address and size have not changed :  
in Linux\_for\_Tegra/source/hardware/nvidia/t23x/nv-public :

```auto
diff --git a/include/kernel/dt-bindings/interrupt-controller/arm-gic.h b/include/kernel/dt-bindings/interrupt-controller/arm-gic.h
index 35b6f69..b11d901 100644
--- a/include/kernel/dt-bindings/interrupt-controller/arm-gic.h
+++ b/include/kernel/dt-bindings/interrupt-controller/arm-gic.h
@@ -20,4 +20,8 @@
 #define GIC_CPU_MASK_RAW(x) ((x) << 8)
 #define GIC_CPU_MASK_SIMPLE(num) GIC_CPU_MASK_RAW((1 << (num)) - 1)
 
+/* SPIs which are available for PCIe MSI interrupts */
+#define GIC_SPI_MSI_BASE 608
+#define GIC_SPI_MSI_SIZE 70
+
 #endif

```

Attempt 1)

```auto
diff --git a/tegra234.dtsi b/tegra234.dtsi
index dad3f53..cc2cb7e 100644
--- a/tegra234.dtsi
+++ b/tegra234.dtsi
@@ -1846,6 +1846,17 @@
                        #redistributor-regions = <1>;
                        #interrupt-cells = <3>;
                        interrupt-controller;
+ ranges; /* v2m fails to allocate without this */
+
+ gic_v2m: v2m@f410000 {
+ compatible = "arm,gic-v2m-frame";
+ msi-controller;
+ #msi-cells = <1>;
+ reg = <0x0 0x0f410000 0x0 0x00010000 /* GICA */
+ 0x0 0x54000000 0x0 0x04000000>;
+ reg-names = "gic_base", "msi_base";
+ arm,msi-base-spi = <GIC_SPI_MSI_BASE>;
+ arm,msi-num-spis = <GIC_SPI_MSI_SIZE>;
+ };
                };
 
                smmu_iso: iommu@10000000 {
@@ -2487,6 +2498,8 @@
                        interconnect-names = "dma-mem", "write";
                        iommu-map = <0x0 &smmu_niso1 TEGRA234_SID_PCIE1 0x1000>;
                        iommu-map-mask = <0x0>;
+ msi-parent = <&gic_v2m TEGRA234_SID_PCIE1>;
+ msi-map = <0x0 &gic_v2m TEGRA234_SID_PCIE1 0x1000>;
                        dma-coherent;
 
                        status = "disabled";

```

No success. The system fails to allocate MSIs at startup.

I also tried with enabling ITS MSI controller of GICv3 as per the [documentation.](https://www.kernel.org/doc/Documentation/devicetree/bindings/interrupt-controller/arm%2Cgic-v3.txt). Same issue at allocation.

Attempt 2)

```auto
diff --git a/tegra234.dtsi b/tegra234.dtsi
index dad3f53..8e7b344 100644
--- a/tegra234.dtsi
+++ b/tegra234.dtsi
@@ -1846,6 +1846,17 @@
                        #redistributor-regions = <1>;
                        #interrupt-cells = <3>;
                        interrupt-controller;
+ msi-controller;
+ mbi-ranges = <GIC_SPI_MSI_BASE GIC_SPI_MSI_SIZE>;
+
+ gic_its: gic-its@f410000 {
+ compatible = "arm,gic-v3-its";
+ msi-controller;
+ #msi-cells = <1>;
+ reg = <0x0 0x0f410000 0x0 0x00010000 /* GICA */
+ 0x0 0x54000000 0x0 0x04000000>;
+ reg-names = "gic_base", "msi_base";
+ };
                };
 
                smmu_iso: iommu@10000000 {
@@ -2487,6 +2498,8 @@
                        interconnect-names = "dma-mem", "write";
                        iommu-map = <0x0 &smmu_niso1 TEGRA234_SID_PCIE1 0x1000>;
                        iommu-map-mask = <0x0>;
+ msi-parent = <&gic_its TEGRA234_SID_PCIE1>;
+ msi-map = <0x0 &gic_its TEGRA234_SID_PCIE1 0x1000>;
                        dma-coherent;
 
                        status = "disabled";

```

I have tried a couple of other options without success.  
Note: I have only modified, recompiled the dtbs and changed FDT entry in extlinux.conf but I did not rebuild the kernel in these experiments.

Seems to me that this is **either a regression or a missing feature in R36.2 DP.**  
This affects IRQ affinity of nvme and other PCI-MSI devices.

In our case, system is throttling because of all devices having IRQs on CPU0.

Thank you for your assistance.

**Update:** after rebuilding the kernel with 1) and added ranges property, MSI are allocated and V2m shows properly configured in dmesg:

```auto
[0.000000] GICv2m: DT overriding V2M MSI_TYPER (base:608, num:70)
[0.000000] GICv2m: range[mem 0x0f410000-0x0f41ffff], SPI[608:677]

```

however with this error:

```auto
[71.210969] tegra-mc 2c00000.memory-controller: pcie1w: write @0x000000000f410040: EMEM address decode error (EMEM decode error)
[71.218804] tegra-mc 2c00000.memory-controller: pcie1w: write @0x000000000f410040: EMEM address decode error (EMEM decode error)

```

Hinted by this log from R35 not being present in R36:

```auto
[6.083493] tegra194-pcie 14100000.pcie: Using GICv2m MSI allocator
[6.091030] tegra194-pcie 14160000.pcie: Adding to iommu group 8

```

I dug into `drivers/pci/controller/dwc/pcie-tegra194.c` and the new upstreamed R36 version does not use GICv2m as an MSI allocator if possible (functions related to `tegra_pcie_parse_msi_parent` do not exist anymore). _So no chances to get this to work this way._

**Update 2** : This seems to be related to [this](https://patchwork.kernel.org/project/linux-omap/cover/20200924190421.549cb8fc@xhacker.debian/#23643455) 2020 discussion with Vidya Sagar from Nvidia and author of pcie-tegra194.c module.

---

_[View the full topic](https://forums.developer.nvidia.com/t/orin-nx-r36-2-cannot-change-irq-smp-affinity/280988)._