Frequent “Dyn index -1 setting as unhealthy” and SFN/Slot mismatches between L2 and L1 in Aerial + OAI integration

furuki-takuma409 · October 24, 2025, 10:55am

While running Aerial cuBB with OAI gNB integration, we frequently observe repeated log messages such as:

nv-cubb | 10:40:19.990414 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 98.19 Slot Map 97 cell index 0 Dyn index -1 setting as healthy!

nv-cubb | 10:40:20.011964 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 100.19 Slot Map 99 cell index 0 Dyn index -1 setting as unhealthy!

c_oai-gnb-aerial | [NR_MAC] Frame.Slot 128.0

c_oai-gnb-aerial |

nv-cubb | 10:40:20.760010 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 24000

nv-cubb | 10:40:21.010413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 200.19 Slot Map 199 cell index 0 Dyn index -1 setting as healthy!

nv-cubb | 10:40:21.031962 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 202.19 Slot Map 201 cell index 0 Dyn index -1 setting as unhealthy!

c_oai-gnb-aerial | [NR_MAC] Frame.Slot 256.0

c_oai-gnb-aerial |

nv-cubb | 10:40:21.760010 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 26000

nv-cubb | 10:40:22.030413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 302.19 Slot Map 301 cell index 0 Dyn index -1 setting as healthy!

nv-cubb | 10:40:22.051962 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 304.19 Slot Map 303 cell index 0 Dyn index -1 setting as unhealthy!

nv-cubb | 10:40:22.760010 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 28000

c_oai-gnb-aerial | [NR_MAC] Frame.Slot 384.0

c_oai-gnb-aerial |

nv-cubb | 10:40:23.050414 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 404.19 Slot Map 403 cell index 0 Dyn index -1 setting as healthy!

nv-cubb | 10:40:23.071957 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 406.19 Slot Map 405 cell index 0 Dyn index -1 setting as unhealthy!

nv-cubb | 10:40:23.759512 CON timer_thread 0 [NVLOG.STAT] [TICK.ERROR][N=10000][MIN=1871 AVG=6863 MAX=15287] exceed limit

nv-cubb |

nv-cubb | 10:40:23.760009 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 30000

nv-cubb | 10:40:24.070414 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 506.19 Slot Map 505 cell index 0 Dyn index -1 setting as healthy!

c_oai-gnb-aerial | [NR_MAC] Frame.Slot 512.0

c_oai-gnb-aerial |

nv-cubb | 10:40:24.091960 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 508.19 Slot Map 507 cell index 0 Dyn index -1 setting as unhealthy!

c_oai-gnb-aerial | 619441.546438 [E] 3230200032: vnf_nr_handle_error_indication: vnf_nr_handle_error_indication: Unpack message failed, ignoring

c_oai-gnb-aerial | [NFAPI_VNF]

c_oai-gnb-aerial | ============================================================================

c_oai-gnb-aerial | sfn slot doesn’t match unpacked one! L2->L1 567.12 vs L1->L2 567.14

c_oai-gnb-aerial | ============================================================================

nv-cubb | 10:40:24.675015 CON timer_thread 0 [L2A.MODULE] tick_received: tick_err 15267

nv-cubb | 10:40:24.675016 ERR timer_thread 0 [AERIAL_L2ADAPTER_EVENT] [SCF.PHY] Send LATE SLOT error indication SFN=567 slot=13

nv-cubb | 10:40:24.675017 ERR timer_thread 0 [AERIAL_NVIPC_API_EVENT] [SCF.PHY] Send Err.ind for SFN 567.13 cell_id=0 msg_id=0x82 err_code=0x34

nv-cubb | 10:40:24.760012 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 32000

nv-cubb | 10:40:25.090413 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 608.19 Slot Map 95 cell index 0 Dyn index -1 setting as healthy!

nv-cubb | 10:40:25.111962 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 610.19 Slot Map 97 cell index 0 Dyn index -1 setting as unhealthy!

c_oai-gnb-aerial | [NR_MAC] Frame.Slot 640.0

c_oai-gnb-aerial |

nv-cubb | 10:40:25.760008 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 34000

nv-cubb | 10:40:26.110416 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 710.19 Slot Map 197 cell index 0 Dyn index -1 setting as healthy!

nv-cubb | 10:40:26.131962 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 712.19 Slot Map 199 cell index 0 Dyn index -1 setting as unhealthy!

c_oai-gnb-aerial | [NR_MAC] Frame.Slot 768.0

c_oai-gnb-aerial |

nv-cubb | 10:40:26.760009 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 36000

nv-cubb | 10:40:27.130412 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 812.19 Slot Map 299 cell index 0 Dyn index -1 setting as healthy!

nv-cubb | 10:40:27.151960 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 814.19 Slot Map 301 cell index 0 Dyn index -1 setting as unhealthy!

nv-cubb | 10:40:27.760006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 38000

c_oai-gnb-aerial | [NR_MAC] Frame.Slot 896.0

PTP sync between GM, gNB server, and RU has been confirmed.
However, the connection between RU and DU cannot be confirmed.
How can I resolve the error in this log? Also, how do I connect LiteOn’s RU to Aerial?
I have confirmed that dst_mac_address is configured correctly.

bkecicioglu · October 24, 2025, 5:02pm

Hi @furuki-takuma409,

Welcome to the Aerial forum!

It seems there are problems with the timer thread on your system.

10:40:23.759512 [TICK.ERROR][N=10000][MIN=1871 AVG=6863 MAX=15287] exceed limit
10:40:24.675015 tick_received: tick_err 15267
10:40:24.675016 Send LATE SLOT error indication SFN=567 slot=13
...
c_oai-gnb-aerial | sfn slot doesn't match unpacked one! L2->L1 567.12 vs L1->L2 567.14

Can you make sure there are no other processes running on the core assigned to the timer thread? It is important that the timer thread should provide accurate timing reference to the system.

The logs about cell health and zero UL throughput indicate either nothing is received from the O-RU or not all of the packets are received. It is very likely that this is a symptom.

Thank you.

furuki-takuma409 · October 27, 2025, 7:13am

Thank you for your reply.
My ptp log is following:

nakaolab@nakaolab:~/openairinterface5g/ci-scripts/yaml_files/sa_gh_gnb$ systemctl status ptp4l.service phc2sys.service

● ptp4l.service - Precision Time Protocol (PTP) service
 Loaded: loaded (/usr/lib/systemd/system/ptp4l.service; **enabled**; preset: **enabled**)

 Active: **active (running)** since Mon 2025-10-27 07:05:28 UTC; 28s ago

   Docs: man:ptp4l

Process: 3054201 ExecStartPre=ifconfig aerial00 up (code=exited, status=0/SUCCESS)

Process: 3054203 ExecStartPre=ethtool --set-priv-flags aerial00 tx_port_ts on (code=exited, status=0/SUCCESS)

Process: 3054205 ExecStartPre=ethtool -A aerial00 rx off tx off (code=exited, status=0/SUCCESS)

Process: 3054207 ExecStartPre=ifconfig aerial01 up (code=exited, status=0/SUCCESS)

Process: 3054209 ExecStartPre=ethtool --set-priv-flags aerial01 tx_port_ts on (code=exited, status=0/SUCCESS)

Process: 3054211 ExecStartPre=ethtool -A aerial01 rx off tx off (code=exited, status=0/SUCCESS)
Main PID: 3054213 (ptp4l)
  Tasks: 1 (limit: 146820)

 Memory: 5.2M (peak: 10.1M)

    CPU: 42ms

 CGroup: /system.slice/ptp4l.service

         └─3054213 /usr/sbin/ptp4l -f /etc/ptp.conf
Oct 27 07:05:46 nakaolab ptp4l[3054213]: [865763.859] rms 3 max 5 freq -14022 +/- 12 delay 38468 +/- 0

Oct 27 07:05:47 nakaolab ptp4l[3054213]: [865764.859] rms 2 max 5 freq -14019 +/- 11 delay 38468 +/- 1

Oct 27 07:05:48 nakaolab ptp4l[3054213]: [865765.859] rms 2 max 4 freq -14015 +/- 8 delay 38467 +/- 0

Oct 27 07:05:49 nakaolab ptp4l[3054213]: [865766.859] rms 3 max 6 freq -14022 +/- 13 delay 38468 +/- 0

Oct 27 07:05:50 nakaolab ptp4l[3054213]: [865767.859] rms 2 max 6 freq -14017 +/- 8 delay 38468 +/- 0

Oct 27 07:05:51 nakaolab ptp4l[3054213]: [865768.859] rms 3 max 6 freq -14015 +/- 15 delay 38468 +/- 1

Oct 27 07:05:52 nakaolab ptp4l[3054213]: [865769.859] rms 2 max 4 freq -14020 +/- 11 delay 38468 +/- 0

Oct 27 07:05:53 nakaolab ptp4l[3054213]: [865770.859] rms 3 max 7 freq -14017 +/- 14 delay 38468 +/- 1

Oct 27 07:05:54 nakaolab ptp4l[3054213]: [865771.859] rms 2 max 3 freq -14017 +/- 10 delay 38468 +/- 1

Oct 27 07:05:55 nakaolab ptp4l[3054213]: [865772.859] rms 3 max 4 freq -14022 +/- 12 delay 38468 +/- 0

● phc2sys.service - Synchronize system clock or PTP hardware clock (PHC)
 Loaded: loaded (/usr/lib/systemd/system/phc2sys.service; **enabled**; preset: **enabled**)

 Active: **active (running)** since Mon 2025-10-27 07:05:30 UTC; 26s ago

   Docs: man:phc2sys

Process: 3054214 ExecStartPre=sleep 2 (code=exited, status=0/SUCCESS)
Main PID: 3054216 (sh)
  Tasks: 2 (limit: 146820)

 Memory: 2.1M (peak: 25.1M)

    CPU: 101ms

 CGroup: /system.slice/phc2sys.service

         ├─3054216 /bin/sh -c "taskset -c 41 /usr/sbin/phc2sys -s /dev/ptp\\$(ethtool -T aerial00 | grep PTP | awk '{print \\$4}') -c CLOCK_REALTIME -n 24 -O 0 -R 256 -u 256"

         └─3054223 /usr/sbin/phc2sys -s /dev/ptp0 -c CLOCK_REALTIME -n 24 -O 0 -R 256 -u 256
Oct 27 07:05:46 nakaolab phc2sys[3054223]: [865763.693] CLOCK_REALTIME rms 9 max 20 freq -3456 +/- 39 delay 513 +/- 7

Oct 27 07:05:47 nakaolab phc2sys[3054223]: [865764.710] CLOCK_REALTIME rms 10 max 29 freq -3460 +/- 57 delay 513 +/- 6

Oct 27 07:05:48 nakaolab phc2sys[3054223]: [865765.727] CLOCK_REALTIME rms 12 max 28 freq -3492 +/- 82 delay 514 +/- 7

Oct 27 07:05:49 nakaolab phc2sys[3054223]: [865766.743] CLOCK_REALTIME rms 11 max 28 freq -3449 +/- 74 delay 513 +/- 7

Oct 27 07:05:50 nakaolab phc2sys[3054223]: [865767.760] CLOCK_REALTIME rms 8 max 19 freq -3472 +/- 21 delay 513 +/- 6

Oct 27 07:05:51 nakaolab phc2sys[3054223]: [865768.776] CLOCK_REALTIME rms 8 max 19 freq -3466 +/- 16 delay 514 +/- 8

Oct 27 07:05:52 nakaolab phc2sys[3054223]: [865769.793] CLOCK_REALTIME rms 8 max 20 freq -3465 +/- 27 delay 514 +/- 9

Oct 27 07:05:53 nakaolab phc2sys[3054223]: [865770.810] CLOCK_REALTIME rms 8 max 22 freq -3469 +/- 32 delay 513 +/- 6

Oct 27 07:05:54 nakaolab phc2sys[3054223]: [865771.826] CLOCK_REALTIME rms 9 max 23 freq -3474 +/- 38 delay 513 +/- 6

Oct 27 07:05:55 nakaolab phc2sys[3054223]: [865772.843] CLOCK_REALTIME rms 12 max 27 freq -3449 +/- 76 delay 513 +/- 7

and the core assigned to ptp4l and phc2sys doesn’t has other processes.

CPU 40:
300  40 cpuhp/40

301  40 idle_inject/40

302  40 migration/40

303  40 ksoftirqd/40

304  40 kworker/40:0-events
199565 40 kworker/40:1-slub_flushwq

CPU 41:
307  41 cpuhp/41

308  41 idle_inject/41

309  41 migration/41

310  41 ksoftirqd/41

311  41 kworker/41:0-events

312  41 kworker/41:0H-events_highpri
2288 41 kworker/41:1H

2544 41 kworker/41:1-slub_flushwq

3054213 41 ptp4l

3054223 41 phc2sys

CPU 42:
314  42 cpuhp/42

315  42 idle_inject/42

316  42 migration/42

317  42 ksoftirqd/42

318  42 kworker/42:0-events
199564 42 kworker/42:1-slub_flushwq

If I assign 41 to ptp4l and 42 to phc2sys, logs are not changed.
I think these shows my gNB server is synced.
Could you please check if there’s anything wrong?

bkecicioglu · October 31, 2025, 3:58pm

@furuki-takuma409

Can you please check if there are any interrupts observed on the core that is assigned to the timer thread?

Even if the PTP is synced, if the timer thread is interrupted by other processes, there will be issues with the synchronization of the system.

Thank you.

furuki-takuma409 · November 3, 2025, 7:01am

I need to set kernel policy “real-time“ with tuned-adm, then slot unmatch is disappeared.
Thank you for your advice.

system · November 17, 2025, 7:01am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
AERIAL ARC With Foxconn RPQN-7800E for N78 Aerial Forum aerial-ran-colab	44	439	November 4, 2024
Issue of Slot Mismatching when Testing with ARC-OTA 1.7 [SFN 3.19 Slot Map 2 cell index 0 Dyn index -1 setting as unhealthy!] Aerial Forum ota , aerial-ran-colab	2	91	February 12, 2026
[DRV.FUNC_UL] Slot Map 103, SFN 104 Slot 19 Order kernel timeout error or Exit error for cell index 0 Dyn index -1! Aerial Forum aerial-cuda-accelerated-ran	7	203	July 24, 2024
Aerial L1/ Openair DU L2 FAPI Compatibiity SCF_FAPI_10_04 Aerial Forum aerial-cuda-accelerated-ran , aerial-ran-colab	17	239	August 29, 2024
OAI ran slot ordering Current slot 9 is NOT DL slot, cannot schedule DCI0 for UL data Aerial Forum aerial-cuda-accelerated-ran	1	130	December 5, 2024
Error while integrating cuBB and OAI gNB - Slot 19 Order kernel timeout error Aerial Forum aerial-cuda-accelerated-ran	8	66	March 26, 2026
Segmentation fault at aerial l1 while running with oai l2-l3 Aerial Forum aerial-cuda-accelerated-ran , aerial-ran-colab	27	261	November 20, 2024
Aerial cuBB End-to-End Test Issue Aerial Forum	5	170	April 25, 2025
Mlx5_common: Fail to create MR for address has been occurred Aerial Forum	9	89	November 7, 2025
Running F08 cuPHYController+testmac+rusim fail in one system Aerial Forum aerial-cuda-accelerated-ran	12	107	November 7, 2025

Frequent “Dyn index -1 setting as unhealthy” and SFN/Slot mismatches between L2 and L1 in Aerial + OAI integration

Related topics