# High CPU interrupts limit 10G port's performance to ~5.3Gb/s

**URL:** <https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303>\
**Category:** Jetson AGX Orin\
**Tags:** ethernet\
**Created:** [April 22, 2022, 9:18am UTC](https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303 "2022-04-22T09:18:01Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![rob\_sv2](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@rob\_sv2](https://forums.developer.nvidia.com/u/rob_sv2)\
**Post date:** [April 22, 2022, 9:18am UTC](https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303/1 "2022-04-22T09:18:01Z")

</div>

Hello,  
I ran some iperf3 benchmarks between the Orin and a desktop PC with a 10GbE NIC, and found that when sending packets from the desktop to the Orin, CPU usage for ksoftirqd/0 shot up to 96% and bandwidth was limited to ~5.3Gb/s.  
I gathered from /proc/interrupts every 2 seconds, and also collected data from top.  
[recvtop.txt](https://forums.developer.nvidia.com/uploads/short-url/sPMtjbRczwisOkDFbhqXuqpsMGz.txt) (137.7 KB)  
[recv interrupts.txt](https://forums.developer.nvidia.com/uploads/short-url/ghnpan1Vf627fHsdbhNWtqCXkxj.txt) (897.2 KB)

When instead sending packets from the Orin to the desktop, I was fortunately able to get ~9.4Gb/s with 10% CPU usage from the process irq/297-1-0008.  
[send interrupts.txt](https://forums.developer.nvidia.com/uploads/short-url/ddBWz3W7nDxVX65JH8POAY9y6Ur.txt) (299.1 KB)  
[sendtop.txt](https://forums.developer.nvidia.com/uploads/short-url/f5mKrYQkYHxBRVZ3r8af9qCFgkB.txt) (138.3 KB)

Are these values as expected? I imagine performance would be reduced further for carrier boards that have additional 10GbE ports.

---

<div class="post-metadata">

**Author:** ![WayneWWW](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/waynewww/32/14043_2.png) [@WayneWWW](https://forums.developer.nvidia.com/u/WayneWWW)\
**Post date:** [April 25, 2022, 3:07am UTC](https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303/3 "2022-04-25T03:07:00Z")

</div>

Could you provide the steps you reproduced this issue?

---

<div class="post-metadata">

**Author:** ![rob\_sv2](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@rob\_sv2](https://forums.developer.nvidia.com/u/rob_sv2)\
**Post date:** [April 25, 2022, 3:34am UTC](https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303/4 "2022-04-25T03:34:06Z")

</div>

I connected the Orin Dev Kit’s 10GbE port to an x86-64 desktop PC with its own 10GbE port, then used iperf3 to run some benchmarks.  
To test the Orin’s maximum receive bandwidth, I ran `iperf3 -s` on the Orin and `iperf3 -c [orin IP]` on the desktop.  
To test the Orin’s maximum send bandwidth, I ran `iperf3 -s` on the desktop and `iperf3 -c p [desktop IP]` on the Orin.  
I used `top` to view CPU usage and `cat /proc/interrupts` to view the interrupt counters.

---

<div class="post-metadata">

**Author:** ![Vinson.Chau](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Vinson.Chau](https://forums.developer.nvidia.com/u/Vinson.Chau)\
**Post date:** [May 7, 2022, 7:18am UTC](https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303/7 "2022-05-07T07:18:25Z")

</div>

I have the same problem. Two Orin Dev Kits connect to each other. Iperf benchmark is only 5Gb/s. CPU 1 usage 90%.  
 ![1651907822(1)](https://global.discourse-cdn.com/nvidia/original/3X/6/c/6c98803467089ab3dd95dd50e21c881a94e5117b.jpeg)

---

<div class="post-metadata">

**Author:** ![zorawar.ehan](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@zorawar.ehan](https://forums.developer.nvidia.com/u/zorawar.ehan)\
**Post date:** [May 15, 2022, 5:53pm UTC](https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303/8 "2022-05-15T17:53:30Z")

</div>

Hello, I found the same problem, did you find a solution?

---

<div class="post-metadata">

**Author:** ![zorawar.ehan](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@zorawar.ehan](https://forums.developer.nvidia.com/u/zorawar.ehan)\
**Post date:** [May 15, 2022, 7:47pm UTC](https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303/9 "2022-05-15T19:47:28Z")

</div>

Hello, I found the same problem, did you find a solution?

---

<div class="post-metadata">

**Author:** ![rob\_sv2](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@rob\_sv2](https://forums.developer.nvidia.com/u/rob_sv2)\
**Post date:** [May 24, 2022, 2:02am UTC](https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303/10 "2022-05-24T02:02:34Z")

</div>

No solutions yet.

@WayneWWW  
Is this issue being tracked? What bandwidth has Nvidia measured during internal tests, including with multiple 10GbE interfaces in parallel?

---

<div class="post-metadata">

**Author:** ![riyaadha](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@riyaadha](https://forums.developer.nvidia.com/u/riyaadha)\
**Post date:** [June 15, 2022, 1:15pm UTC](https://forums.developer.nvidia.com/t/high-cpu-interrupts-limit-10g-ports-performance-to-5-3gb-s/212303/11 "2022-06-15T13:15:24Z")

</div>

I am having the same problem. Any update on this issue would be appreciated.
