Jetson AGX Xavier IRQ over GPIO CPU affinity can’t be changed

[venu1]

Hi,

I’m using the Jetson AGX as a root port and a nVIDIA dev board as GPIO endpoint.

I’m able to measured the Interrupt Latency between two gpio pins is well.

By default, IRQ affinity is setup to the CPU0 core of the Jetson AGX. Here is a view of the IRQ associated with my GPIO driver :

cat /proc/interrupts | grep gpio
247: 0 0 0 0 0 0 0 0 tegra-gpio 48 Edge force-recovery
251: 2 0 0 0 0 0 0 0 tegra-gpio 52 level phy_interrupt
254: 0 0 0 0 0 0 0 0 tegra-gpio 55 Edge 3400000.sdhci cd
257: 2 0 0 0 0 0 0 0 tegra-gpio 58 level tmp451
297: 0 0 0 0 0 0 0 0 tegra-gpio 98 Edge 15200000.nvdisplay
348: 0 0 0 0 0 0 0 0 tegra-gpio 149 Edge rt5659
391: 271811308 0 0 0 0 0 0 0 tegra-gpio 192 Edge bluetooth hostwake
436: 22 0 0 0 0 0 0 0 tegra-gpio-aon 9 Edge my_interrupt
437: 19 0 0 0 0 0 0 0 tegra-gpio-aon 10 level ccg_irq
459: 0 0 0 0 0 0 0 0 tegra-gpio-aon 32 Edge fs_cd_gpio
463: 0 0 0 0 0 0 0 0 tegra-gpio-aon 36 Edge power-key
500: 0 0 0 0 0 0 0 0 max77620-top 3 Edge max77620-gpio

The user IRQ is the #436 and I’d like to change its affinity to another CPU core. But the following command return an error :

root@nvidia:/home/nvidia# echo 4 > /proc/irq/436/smp_affinity
bash: echo: write error: Input/output error

I also try to change affinity in Kernel space through the driver adding the following command :
irq_set_affinity_hint(irq, cpumask_of(4));
But it doesn’t work either…

I’ve built the Kenerl and cross-compiled. I’m not getting any errors and warnings and it is the case.

How can I change these IRQ affinity ?

Thanks & Regards,
M. Venu Gopal

[linuxdev]

Please note that not all hardware IRQs can be routed to all CPU cores. I don’t know if this is the case you are running into, but it probably is. There is an IRQ aggregator which routes most hardware IRQs to CPU 0. Being able to route to other cores requires additional IO-APIC hardware.

Someone else would need to confirm if this is the case, but I suspect it is. If I am correct, then even if you could name affinity to the other core, the process would still end up migrating back to CPU 0. I just don’t know enough about the internal IRQ aggregator to say with certainty.

[JerryChang]

hello venu1,

we’ll need to check this internally,
please share more details for reference, may I know which JetPack release you’re working with.
you may check release tag, i.e. $ cat /etc/nv_tegra_release for release version.
thanks

[Bibek]

the gpio ( main and aon) are handled by gpio interrupt controller which collate the interrupts and send it to various CPU and R5. So affinity can be only set to vGIC interrupts. and not GPIO interrupts.

[linuxdev]

This is just curiosity, but I am looking at “/proc/interrupts”, and see a large number of interrupts work through GICv2 (vGIC). A few special purpose buttons (like force-recovery) do not use vGIC, and thus can work even if vGIC is not running. Does this mean that every interrupt listed in “/proc/interrupts” using GICv2 can individually be set for affinity on any core without exception?

The Cortex-R5 is not shown in “/proc/interrupts”, so I am curious what the role of that R5 is. I know there are the Audio Processing Engine (APE) and Signal Processing Engine (SPE) R5s, but I am curious if there are any documents on the R5 specific to the interrupts of the above mentioned IRQ collating.

[prasanna]

Hi Jerry,

Below is the Tegra version :

R32 (release), REVISION: 1.0, GCID: 14531094, BOARD: t186ref, EABI: aarch64, DATE: Wed Mar 13 07:41:08 UTC 2019

So, As per Bibek comment can’t we set the affinity to interrupts generated by GPIO ?

Requirement : Need to handle 2 GPIO’s connected to Jetson from FPGA must be handled separately by two different CPU core.
For Example: GPIO_INTR-1 should be handled in Core-5 and GPIO_INTR-2 in Core-6.
As we need to handle two different GPIO interrupts in two different cores. So tried to set the CPU affinity to interrupts. But could not able to achieve.
If interrupt affinity is not possible in Jetson. What would be your suggestion to achieve the above requirement in the best way.

1. If we use UART interface is there any method to recv interrupt to a particular Core. When UART Txn occur ?
2. Is there any way we can map GPIO to vGIC controller via any dedicated IO of vGIC controller.

Thanks
Prasanna

[JerryChang]

hello prasanna,

FYI,
affinity can only be set at the GIC level.

[prasanna]

Hi Jerry,

Thanks.
UART and PCIe interrupts are routed to GIC. Correct ?
So for UART interrupts I can set affinity ?

Regards,
Prasanna

[JerryChang]

hello prasanna,

are you actually only care about “2” GPIOs. you may choose 1 GPIO from the main GPIO controller and 1 GPIO from the AON GPIO controller.
these 2 controllers have separate GIC IRQs, and so affinity can be set separately.

[prasanna]

Hi Jerry,

Thanks for the inputs.
Actually I need to handle 4 GPIO Interrupts from external hardware in 4 different cores individually and based on interrupts I need to take some actions. But 2 GPIO handling in two different cores also fine for me.
As you suggested I have selected one GPIO from Tegra-Gpio-Aon which is GPIO9_CAN1_GPIO0_DMIC_CLK (GPIO 257). Pin No 32 on 40 pin header.
And I am trying to set the smp_affinity of the corresponding interrupt. But I am getting write error. Please find below screen shot.

Please let me know any other procedure to be followed to achieve. or Any document to be referred ?

As well I observed that interrupt is generated on both the edges. Although I have configured only for rising edge.
Below is the snippet from kernel module that I used.
if (request_irq(irq, my_interrupt_handler, IRQF_TRIGGER_RISING, “my_interrupt”, &my_dev_id))

Please let me know If I missed anything.

Thanks,
Prasanna

[JerryChang]

hello prasanna,

the GPIO interrupt at the GIC is combination of multiple pins from GPIO controller.
so,
if you want to achieve the different affinity for pin level of same GPIO controller then you will need the GPIO routing to one of 8 GPIO controller’s interrupt line and then that interrupt line to the particular CPU via GICs.