Mlx5_common: Fail to create MR for address has been occurred

furuki-takuma409 · October 31, 2025, 12:53pm

I’m trying to connect liteon RU and oai+aerial gnb.

then I got following error

nv-cubb | mlx5_common: Fail to create MR for address (0xe2ede0000000)

nv-cubb | mlx5_common: Device 0000:01:00.0 unable to DMA map

nv-cubb | 05:55:42.068259 CON phy_init 0 [FH.PEER] Doca sem created!: sem_in gpu_addr 0xe301ce3a0000 cpu_addr 0x42e7b375f80

nv-cubb | 05:55:42.129380 ERR phy_init 0 [AERIAL_ORAN_FH_EVENT] [FH.LIB] Exception! Failed to DMA map addr 0xe2ede0000000 for NIC 0000:01:00.0: Operation not permitted

nv-cubb | 05:55:42.129396 ERR phy_init 0 [AERIAL_ORAN_FH_EVENT] [DRV.FH] aerial_fh::register_memory error

nv-cubb | 05:55:42.129397 ERR phy_init 0 [AERIAL_CUPHYDRV_API_EVENT] [DRV.CTX] registerMem error

What does this error mean?

Should I deal with this error?

Following errors has also been occurred

nv-cubb | 05:56:39.990415 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 498.19 Slot Map 497 cell index 0 Dyn index -1 setting as healthy!

nv-cubb | 05:56:40.011966 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 500.19 Slot Map 499 cell index 0 Dyn index -1 setting as unhealthy!

20251031_gnb.log (64.6 KB)

bkecicioglu · October 31, 2025, 4:51pm

Hi @furuki-takuma409 ,

Here is the origin of the errors

MLX5 driver fails to create Memory Region (MR) for GPU address
NIC cannot DMA map the GPU memory
Aerial FH library fails to register the memory
cuPHYDriver fails to register memory with the fronthaul driver

It is likely that

GPUDirect RDMA not enabled or configured,
Have insufficient privileges

Can you please share the output of the following commands?

# Check if nvidia_peermem is loaded
lsmod | grep nvidia_peermem

# Check MLNX_OFED installation
ofed_info -s

# Check GPU peer access capabilities
nvidia-smi topo -m

# Check process capabilities
getcap /path/to/your/binary

The second batch of errors indicate no packets are received from the FH for UL data. This is the meaning of cell being “unhealthy”.

Thank you.

furuki-takuma409 · November 3, 2025, 6:43am

After loading the nvidia_peermem module, the previous DMA mapping error has disappeared.
Now, I can see that packets from the O-RU are successfully arriving at the DU (verified with Wireshark).
However, the DU still reports the cell as “unhealthy” and UL 0bps, even though the U-plane packets are visible on the interface.

Do you have any suggestions on additional steps?

Thank you for your help and guidance.

Cannot find MPS control daemon process

Supermicro-G1SMH-G
Started cuphycontroller on CPU core 0
Aerial metrics backend address: 127.0.0.1:8081
06:15:43.989940 CON phy_init 0 [CTL.SCF] Config file: /opt/nvidia/cuBB/cuPHY-CP/cuphycontroller/config/cuphycontroller_P5G_FXN_GH.yaml
06:15:43.990941 CON phy_init 0 [CTL.SCF] low_priority_core=10
06:15:43.990955 CON phy_init 0 [APP.CONFIG] Current TAI offset: 0s
06:15:43.990992 CON phy_init 0 [NVLOG.CPP] Using /opt/nvidia/cuBB/cuPHY/nvlog/config/nvlog_config.yaml for nvlog configuration
06:15:43.992179 CON phy_init 0 [NVLOG.CPP] AERIAL_LOG_PATH set to /var/log/aerial

06:15:43.992182 CON phy_init 0 [NVLOG.CPP] Output log file path /var/log/aerial/phy.log
06:15:44.003558 CON phy_init 0 [CTL.YAML] cuphycontroller config. yaml does not have gpu_init_comms_via_cpu key; defaulting to 0.
06:15:44.003559 CON phy_init 0 [CTL.YAML] cuphycontroller config. yaml does not have cpu_init_comms key; defaulting to 0.
06:15:44.003694 CON phy_init 0 [CTL.YAML] cell_id 1 nic_index :0
06:15:44.003788 CON phy_init 0 [CTL.YAML] ta4_min_ns_srs/ta4_max_ns_srs not set in config file, using default ta4_min_ns_srs = 621 and ta4_max_ns_srs = 1831
06:15:44.003802 CON phy_init 0 [CTL.YAML] t1a_min_cp_dl_ns/t1a_max_cp_dl_ns not set in config file, using default t1a_min_cp_dl_ns = 419 and t1a_max_cp_dl_ns = 669
06:15:44.003806 CON phy_init 0 [CTL.YAML] t1a_min_cp_ul_ns not set in config file, using default of 285 us
06:15:44.003845 CON phy_init 0 [CTL.YAML] Num Slots: 8
06:15:44.003845 CON phy_init 0 [CTL.YAML] Enable UL cuPHY Graphs: 1
06:15:44.003845 CON phy_init 0 [CTL.YAML] Enable DL cuPHY Graphs: 1
06:15:44.003846 CON phy_init 0 [CTL.YAML] Accurate TX scheduling clock resolution (ns): 500
06:15:44.003846 CON phy_init 0 [CTL.YAML] DPDK core: 10
06:15:44.003846 CON phy_init 0 [CTL.YAML] Prometheus core: -1
06:15:44.003847 CON phy_init 0 [CTL.YAML] UL cores:
06:15:44.003847 CON phy_init 0 [CTL.YAML] - 4
06:15:44.003847 CON phy_init 0 [CTL.YAML] - 5
06:15:44.003847 CON phy_init 0 [CTL.YAML] DL cores:
06:15:44.003847 CON phy_init 0 [CTL.YAML] - 6
06:15:44.003847 CON phy_init 0 [CTL.YAML] - 7
06:15:44.003847 CON phy_init 0 [CTL.YAML] - 8
06:15:44.003847 CON phy_init 0 [CTL.YAML] Debug worker: -1
06:15:44.003848 CON phy_init 0 [CTL.YAML] Data Lake core: -1
06:15:44.003848 CON phy_init 0 [CTL.YAML] SRS starting Section ID: 3072
06:15:44.003848 CON phy_init 0 [CTL.YAML] PRACH starting Section ID: 2048
06:15:44.003849 CON phy_init 0 [CTL.YAML] USE GREEN CONTEXTS: 0
06:15:44.003849 CON phy_init 0 [CTL.YAML] USE BATCHED MEMCPY: 1
06:15:44.003849 CON phy_init 0 [CTL.YAML] MPS SM PUSCH: 100
06:15:44.003849 CON phy_init 0 [CTL.YAML] MPS SM PUCCH: 2
06:15:44.003849 CON phy_init 0 [CTL.YAML] MPS SM PRACH: 2
06:15:44.003849 CON phy_init 0 [CTL.YAML] MPS SM UL ORDER: 20
06:15:44.003849 CON phy_init 0 [CTL.YAML] MPS SM PDSCH: 102
06:15:44.003849 CON phy_init 0 [CTL.YAML] MPS SM PDCCH: 10
06:15:44.003850 CON phy_init 0 [CTL.YAML] MPS SM PBCH: 2
06:15:44.003850 CON phy_init 0 [CTL.YAML] MPS SM GPU_COMMS: 16
06:15:44.003850 CON phy_init 0 [CTL.YAML] MPS SM SRS: 16
06:15:44.003850 CON phy_init 0 [CTL.YAML] UL Order Kernel Mode: 0
06:15:44.003850 CON phy_init 0 [CTL.YAML] PDSCH fallback: 0
06:15:44.003851 CON phy_init 0 [CTL.YAML] Massive MIMO enable: 0
06:15:44.003851 CON phy_init 0 [CTL.YAML] mMIMO_enable feature 0
06:15:44.003851 CON phy_init 0 [CTL.YAML] Enable SRS : 1
06:15:44.003851 CON phy_init 0 [CTL.YAML] ul_order_timeout_gpu_log_enable: 0
06:15:44.003852 CON phy_init 0 [CTL.YAML] ue_mode: 0
06:15:44.003852 CON phy_init 0 [CTL.YAML] Aggr Obj Non-availability threshold: 5
06:15:44.003852 CON phy_init 0 [CTL.YAML] sendCPlane_timing_error_th_ns: 0
06:15:44.003852 CON phy_init 0 [CTL.YAML] pusch_aggr_per_ctx: 3
06:15:44.003852 CON phy_init 0 [CTL.YAML] prach_aggr_per_ctx: 2
06:15:44.003852 CON phy_init 0 [CTL.YAML] pucch_aggr_per_ctx: 4
06:15:44.003852 CON phy_init 0 [CTL.YAML] srs_aggr_per_ctx: 3
06:15:44.003852 CON phy_init 0 [CTL.YAML] max_harq_pools: 384
06:15:44.003852 CON phy_init 0 [CTL.YAML] ul_input_buffer_per_cell: 10
06:15:44.003852 CON phy_init 0 [CTL.YAML] ul_input_buffer_per_cell_srs: 6
06:15:44.003852 CON phy_init 0 [CTL.YAML] max_ru_unhealthy_ul_slots: 100
06:15:44.003852 CON phy_init 0 [CTL.YAML] srs_chest_algo_type: 0
06:15:44.003852 CON phy_init 0 [CTL.YAML] srs_chest_tol2_normalization_algo_type: 1
06:15:44.003853 CON phy_init 0 [CTL.YAML] srs_chest_tol2_constant_scaler: 32768
06:15:44.003853 CON phy_init 0 [CTL.YAML] bfw_power_normalization_alg_selector: 1
06:15:44.003853 CON phy_init 0 [CTL.YAML] bfw_beta_prescaler: 16384
06:15:44.003853 CON phy_init 0 [CTL.YAML] total_num_srs_chest_buffers: 6144
06:15:44.003853 CON phy_init 0 [CTL.YAML] send_static_bfw_wt_all_cplane: 1
06:15:44.003853 CON phy_init 0 [CTL.YAML] ul_pcap_capture_enable: 0
06:15:44.003853 CON phy_init 0 [CTL.YAML] ul_pcap_capture_thread_cpu_affinity: 117
06:15:44.003853 CON phy_init 0 [CTL.YAML] ul_pcap_capture_thread_sched_priority: 175
06:15:44.003853 CON phy_init 0 [CTL.YAML] pcap_logger_ul_cplane_enable: 0
06:15:44.003853 CON phy_init 0 [CTL.YAML] pcap_logger_dl_cplane_enable: 0
06:15:44.003853 CON phy_init 0 [CTL.YAML] pcap_logger_thread_cpu_affinity: 10
06:15:44.003854 CON phy_init 0 [CTL.YAML] pcap_logger_thread_sched_prio: 95
06:15:44.003854 CON phy_init 0 [CTL.YAML] pcap_logger_file_save_dir: ./
06:15:44.003854 CON phy_init 0 [CTL.YAML] static_beam_id_start: 1
06:15:44.003854 CON phy_init 0 [CTL.YAML] static_beam_id_end: 16527
06:15:44.003854 CON phy_init 0 [CTL.YAML] dynamic_beam_id_start: 16528
06:15:44.003854 CON phy_init 0 [CTL.YAML] dynamic_beam_id_end: 65535
06:15:44.003854 CON phy_init 0 [CTL.YAML] ul_order_timeout_gpu_log_enable: 0
06:15:44.003854 CON phy_init 0 [CTL.YAML] pusch_workCancelMode: 2
06:15:44.003854 CON phy_init 0 [CTL.YAML] GPU-initiated comms DL: 1
06:15:44.003855 CON phy_init 0 [CTL.YAML] GPU-initiated comms (via CPU): 0
06:15:44.003855 CON phy_init 0 [CTL.YAML] CPU-initiated comms : 0
06:15:44.003855 CON phy_init 0 [CTL.YAML] Cell group: 1
06:15:44.003855 CON phy_init 0 [CTL.YAML] Cell group num: 1
06:15:44.003855 CON phy_init 0 [CTL.YAML] puxchPolarDcdrListSz: 8
06:15:44.003855 CON phy_init 0 [CTL.YAML] split_ul_cuda_streams: 0
06:15:44.003855 CON phy_init 0 [CTL.YAML] serialize_pucch_pusch: 0
06:15:44.003855 CON phy_init 0 [CTL.YAML] bfw_c_plane_chaining_mode: 2
06:15:44.003855 CON phy_init 0 [CTL.YAML] fh_stats_dump_cpu_core: -1
06:15:44.003855 CON phy_init 0 [CTL.YAML] fix_beta_dl: 0
06:15:44.003856 CON phy_init 0 [CTL.YAML] pdump_client_thread: -1
06:15:44.003856 CON phy_init 0 [CTL.YAML] profiler_sec: 0
06:15:44.003856 CON phy_init 0 [CTL.YAML] datalake_address: localhost
06:15:44.003856 CON phy_init 0 [CTL.YAML] dpdk_file_prefix: cuphycontroller
06:15:44.003856 CON phy_init 0 [CTL.YAML] puschrxChestFactorySettingsFilename:
06:15:44.003856 CON phy_init 0 [CTL.YAML] accu_tx_sched_disable: 0
06:15:44.003856 CON phy_init 0 [CTL.YAML] cplane_disable: 0
06:15:44.003856 CON phy_init 0 [CTL.YAML] disable_empw: 0
06:15:44.003856 CON phy_init 0 [CTL.YAML] dlc_alloc_cplane_bfw_txq: 0
06:15:44.003856 CON phy_init 0 [CTL.YAML] dlc_bfw_enable_divide_per_cell: 0
06:15:44.003857 CON phy_init 0 [CTL.YAML] dpdk_verbose_logs: 0
06:15:44.003857 CON phy_init 0 [CTL.YAML] enable_cpu_task_tracing: 0
06:15:44.003857 CON phy_init 0 [CTL.YAML] enable_dl_cqe_tracing: 0
06:15:44.003857 CON phy_init 0 [CTL.YAML] enable_l1_param_sanity_check: 0
06:15:44.003857 CON phy_init 0 [CTL.YAML] enable_ok_tb: 0
06:15:44.003857 CON phy_init 0 [CTL.YAML] enable_prepare_tracing: 0
06:15:44.003857 CON phy_init 0 [CTL.YAML] mCh_segment_proc_enable: 0
06:15:44.003857 CON phy_init 0 [CTL.YAML] pmu_metrics: 0
06:15:44.003857 CON phy_init 0 [CTL.YAML] puschCfo: 1
06:15:44.003858 CON phy_init 0 [CTL.YAML] puschDftSOfdm: 0
06:15:44.003858 CON phy_init 0 [CTL.YAML] puschEnablePerPrgChEst: 0
06:15:44.003858 CON phy_init 0 [CTL.YAML] puschRssi: 1
06:15:44.003858 CON phy_init 0 [CTL.YAML] puschSelectChEstAlgo: 1
06:15:44.003858 CON phy_init 0 [CTL.YAML] puschSelectEqCoeffAlgo: 3
06:15:44.003858 CON phy_init 0 [CTL.YAML] puschSinr: 2
06:15:44.003858 CON phy_init 0 [CTL.YAML] puschTbSizeCheck: 1
06:15:44.003858 CON phy_init 0 [CTL.YAML] puschTdi: 1
06:15:44.003858 CON phy_init 0 [CTL.YAML] puschTo: 1
06:15:44.003859 CON phy_init 0 [CTL.YAML] pusch_deviceGraphLaunchEn: 1
06:15:44.003859 CON phy_init 0 [CTL.YAML] pusch_subSlotProcEn: 0
06:15:44.003859 CON phy_init 0 [CTL.YAML] ulc_alloc_cplane_bfw_txq: 1
06:15:44.003859 CON phy_init 0 [CTL.YAML] ulc_bfw_enable_divide_per_cell: 0
06:15:44.003859 CON phy_init 0 [CTL.YAML] ul_rx_pkt_tracing_level: 0
06:15:44.003859 CON phy_init 0 [CTL.YAML] ul_rx_pkt_tracing_level_srs: 0
06:15:44.003859 CON phy_init 0 [CTL.YAML] forcedNumCsi2Bits: 0
06:15:44.003859 CON phy_init 0 [CTL.YAML] pusch_waitTimeOutPostEarlyHarqUs: 3000
06:15:44.003859 CON phy_init 0 [CTL.YAML] pusch_waitTimeOutPreEarlyHarqUs: 3000
06:15:44.003859 CON phy_init 0 [CTL.YAML] validation: 0
06:15:44.003860 CON phy_init 0 [CTL.YAML] cqe_trace_slot_mask: 196800
06:15:44.003860 CON phy_init 0 [CTL.YAML] datalake_samples: 1000000
06:15:44.003860 CON phy_init 0 [CTL.YAML] num_ok_tb_slot: 0
06:15:44.003860 CON phy_init 0 [CTL.YAML] pusch_nMaxLdpcHetConfigs: 32
06:15:44.003860 CON phy_init 0 [CTL.YAML] sendCPlane_dlbfw_backoff_th_ns: 300000
06:15:44.003860 CON phy_init 0 [CTL.YAML] sendCPlane_ulbfw_backoff_th_ns: 300000
06:15:44.003860 CON phy_init 0 [CTL.YAML] ul_order_max_rx_pkts: 512
06:15:44.003860 CON phy_init 0 [CTL.YAML] ul_order_rx_pkts_timeout_ns: 100000
06:15:44.003860 CON phy_init 0 [CTL.YAML] ul_order_timeout_cpu_ns: 4000000
06:15:44.003860 CON phy_init 0 [CTL.YAML] ul_order_timeout_gpu_ns: 3000000
06:15:44.003860 CON phy_init 0 [CTL.YAML] ul_order_timeout_gpu_srs_ns: 5200000
06:15:44.003861 CON phy_init 0 [CTL.YAML] ul_order_timeout_log_interval_ns: 1000000000
06:15:44.003861 CON phy_init 0 [CTL.YAML] ul_srs_aggr3_task_launch_offset_ns: 4000000
06:15:44.003861 CON phy_init 0 [CTL.YAML] workers_sched_priority: 95
06:15:44.003861 CON phy_init 0 [CTL.YAML] cqe_trace_cell_mask: 1
06:15:44.003861 CON phy_init 0 [CTL.YAML] Number of Cell Configs: 1
06:15:44.003861 CON phy_init 0 [CTL.YAML] L2Adapter config file: /opt/nvidia/cuBB/cuPHY-CP/cuphycontroller/config/l2_adapter_config_P5G_GH.yaml
06:15:44.003861 CON phy_init 0 [CTL.YAML] Cell name: O-RU 0
06:15:44.003862 CON phy_init 0 [CTL.YAML] MU: 1
06:15:44.003862 CON phy_init 0 [CTL.YAML] ID: 1
06:15:44.003862 CON phy_init 0 [CTL.YAML] Number of MPlane Configs: 1
06:15:44.003862 CON phy_init 0 [CTL.YAML] Mplane ID: 1
06:15:44.003862 CON phy_init 0 [CTL.YAML] VLAN ID: 564
06:15:44.003863 CON phy_init 0 [CTL.YAML] Source Eth Address: a0:88:c2:a9:c0:d0
06:15:44.003863 CON phy_init 0 [CTL.YAML] Destination Eth Address: e8:c7:4f:25:7f:ff
06:15:44.003863 CON phy_init 0 [CTL.YAML] NIC port: 0000:01:00.0
06:15:44.003863 CON phy_init 0 [CTL.YAML] RU Type: 1
06:15:44.003863 CON phy_init 0 [CTL.YAML] U-plane TXQs: 1
06:15:44.003863 CON phy_init 0 [CTL.YAML] DL compression method: 1
06:15:44.003863 CON phy_init 0 [CTL.YAML] DL iq bit width: 9
06:15:44.003863 CON phy_init 0 [CTL.YAML] UL compression method: 1
06:15:44.003864 CON phy_init 0 [CTL.YAML] UL iq bit width: 9
06:15:44.003864 CON phy_init 0 [CTL.YAML]
06:15:44.003864 CON phy_init 0 [CTL.YAML] Flow list SSB/PBCH:
06:15:44.003864 CON phy_init 0 [CTL.YAML] 0
06:15:44.003864 CON phy_init 0 [CTL.YAML] 1
06:15:44.003864 CON phy_init 0 [CTL.YAML] 2
06:15:44.003864 CON phy_init 0 [CTL.YAML] 3
06:15:44.003864 CON phy_init 0 [CTL.YAML] Flow list PDCCH:
06:15:44.003864 CON phy_init 0 [CTL.YAML] 0
06:15:44.003864 CON phy_init 0 [CTL.YAML] 1
06:15:44.003864 CON phy_init 0 [CTL.YAML] 2
06:15:44.003864 CON phy_init 0 [CTL.YAML] 3
06:15:44.003864 CON phy_init 0 [CTL.YAML] Flow list PDSCH:
06:15:44.003864 CON phy_init 0 [CTL.YAML] 0
06:15:44.003865 CON phy_init 0 [CTL.YAML] 1
06:15:44.003865 CON phy_init 0 [CTL.YAML] 2
06:15:44.003865 CON phy_init 0 [CTL.YAML] 3
06:15:44.003865 CON phy_init 0 [CTL.YAML] Flow list CSIRS:
06:15:44.003865 CON phy_init 0 [CTL.YAML] 0
06:15:44.003865 CON phy_init 0 [CTL.YAML] 1
06:15:44.003865 CON phy_init 0 [CTL.YAML] 2
06:15:44.003865 CON phy_init 0 [CTL.YAML] 3
06:15:44.003865 CON phy_init 0 [CTL.YAML] Flow list PUSCH:
06:15:44.003865 CON phy_init 0 [CTL.YAML] 0
06:15:44.003865 CON phy_init 0 [CTL.YAML] 1
06:15:44.003865 CON phy_init 0 [CTL.YAML] 2
06:15:44.003865 CON phy_init 0 [CTL.YAML] 3
06:15:44.003865 CON phy_init 0 [CTL.YAML] Flow list PUCCH:
06:15:44.003866 CON phy_init 0 [CTL.YAML] 0
06:15:44.003866 CON phy_init 0 [CTL.YAML] 1
06:15:44.003866 CON phy_init 0 [CTL.YAML] 2
06:15:44.003866 CON phy_init 0 [CTL.YAML] 3
06:15:44.003866 CON phy_init 0 [CTL.YAML] Flow list SRS:
06:15:44.003866 CON phy_init 0 [CTL.YAML] 0
06:15:44.003866 CON phy_init 0 [CTL.YAML] 1
06:15:44.003866 CON phy_init 0 [CTL.YAML] 2
06:15:44.003866 CON phy_init 0 [CTL.YAML] 3
06:15:44.003866 CON phy_init 0 [CTL.YAML] Flow list PRACH:
06:15:44.003866 CON phy_init 0 [CTL.YAML] 0
06:15:44.003866 CON phy_init 0 [CTL.YAML] 1
06:15:44.003866 CON phy_init 0 [CTL.YAML] 2
06:15:44.003866 CON phy_init 0 [CTL.YAML] 3
06:15:44.003866 CON phy_init 0 [CTL.YAML] PUSCH TV: /opt/nvidia/cuBB/testVectors/cuPhyChEstCoeffs.h5
06:15:44.003867 CON phy_init 0 [CTL.YAML] SRS TV: /opt/nvidia/cuBB/testVectors/cuPhyChEstCoeffs.h5
06:15:44.003867 CON phy_init 0 [CTL.YAML] Section_3 time offset: 58369
06:15:44.003867 CON phy_init 0 [CTL.YAML] nMaxRxAnt: 4
06:15:44.003867 CON phy_init 0 [CTL.YAML] PUSCH PRBs Stride: 273
06:15:44.003867 CON phy_init 0 [CTL.YAML] PRACH PRBs Stride: 12
06:15:44.003867 CON phy_init 0 [CTL.YAML] SRS PRBs Stride: 273
06:15:44.003867 CON phy_init 0 [CTL.YAML] PUSCH nMaxPrb: 273
06:15:44.003867 CON phy_init 0 [CTL.YAML] PUSCH nMaxRx: 4
06:15:44.003867 CON phy_init 0 [CTL.YAML] UL Gain Calibration: 48.68
06:15:44.003868 CON phy_init 0 [CTL.YAML] Lower guard bw: 845
06:15:44.591752 CON phy_init 0 [CTL.SCF] Network interface for PCIe address 0000:01:00.0 : aerial00
06:15:44.591824 CON phy_init 0 [APP.UTILS] PHC clock: 1762150544.591819626
06:15:44.591835 CON phy_init 0 [APP.UTILS] CLOCK_TAI: 1762150544.591829200
06:15:44.591836 CON phy_init 0 [APP.UTILS] CLOCK_REALTIME: 1762150544.591829136
06:15:44.591836 CON phy_init 0 [APP.UTILS] TAI/REALTIME offset: 0 seconds
06:15:44.594803 CON phy_init 0 [DRV.CTX] CUDA_DEVICE_MAX_CONNECTIONS 8
06:15:44.594807 CON phy_init 0 [DRV.CTX] use_green_contexts 0
EAL: Detected CPU lcores: 72
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/cuphycontroller/mp_socket
EAL: Selected IOVA mode ‘VA’
EAL: VFIO support initialized
EAL: Probe PCI driver: gpu_cuda (10de:2342) device: 0009:01:00.0 (socket 0)
EAL: Probe PCI driver: mlx5_pci (15b3:a2dc) device: 0000:01:00.0 (socket 0)
06:15:46.594565 CON phy_init 0 [FH.DOCA] doca_gpu_dmabuf_fd returned 0 (DOCA_SUCCESS is 0)
06:15:46.594577 CON phy_init 0 [FH.DOCA] Mapping transmit queue buffer (0xfdda00000000 size 4294967296B dmabuf fd 219) with dmabuf mode
PCAP_LOGGER Initialized
06:15:46.627652 CON phy_init 0 [FH.PEER] Doca sem created!: sem_in gpu_addr 0xfdec8e3a0000 cpu_addr 0x52ed33f5ee0
gRPC Server listening on 0.0.0.0:50051
06:15:47.121439 CON phy_main 0 [CTL.SCF]
====> PhyDriver initialized!
06:15:47.121662 CON phy_main 0 [NVIPC:YAML] nv_ipc_parse_yaml_node: PCAP msg_filter: enable all messages: 0x00-0xFF
06:15:47.121670 CON phy_main 0 [NVIPC:YAML] nv_ipc_parse_yaml_node: PCAP cell_filter: enable all cells 0-255
06:15:47.121881 CON phy_main 0 [NVIPC.DEBUG] nv_ipc_debug_open: prefix=nvipc fapi_type=1 pcap_sync_save=0 nvipc_version=25.2.0 OK
06:15:47.144948 CON 138 0 [NVIPC.DEBUG] pcap_shm_caching_thread_func: thread [nvipc_pcap_shm] started on CPU core [10] …
06:15:47.169645 CON 138 0 [NVIPC.DEBUG] Open PCAP logger: prefix=nvipc_pcap shm_caching_core=10 file_saving_core=10 max_data_size=8000 cache_size=512MB round_save_size=2048MB
06:15:47.179229 CON phy_main 0 [NVIPC.SHM] nvipc server initialized
06:15:47.179313 CON phy_main 0 [NVIPC.SHM] shm_ipc_open: forward_enable=-1 fw_max_msg_buf_count=1365 fw_max_data_buf_count=341
06:15:47.179318 CON phy_main 0 [NVIPC.SHM] create_shm_nv_ipc_interface: OK. buf_size: cpu_msg=15000 cpu_data=576000 cpu_large=4096000
06:15:47.179329 CON phy_main 0 [L2A.TRANSPORT] phy_mac_transport[0] created. phy_cell_map.size=0 mac_cell_map.size=0 mapped=false cell_num=1
06:15:47.179332 CON phy_main 0 [L2A.TRANSPORT] init: loaded 1 nvipc instance, total_cell_num=1
06:15:47.179363 CON phy_main 0 [L2A.MODULE] PHY_module: transport_num=1 total_cell_num=1
06:15:47.183399 CON 142 0 [NVIPC.EFD] [nvipc][core 10 ] share event_fd thread info: sched_policy=0 sched_priority=0
06:15:47.183457 CON 142 0 [NVIPC.EFD] [nvipc] Listening for nvipc client to connect …
06:15:47.208388 CON phy_main 0 [CTL.SCF] cuPHYController configured for 1 cells
06:15:47.208391 CON phy_main 0 [CTL.SCF]
====> cuPHYController initialized, L1 is ready!
06:15:47.208406 CON phy_main 0 [CTL.STARTUP_TIMES] {TI PERCENTAGE} Start Main:0.3,Parse Cuphycontroller YAML:0.0,Init nvlog:0.4,Cuda Set Device:17.4,Cuphy PTI Init:0.8,Init PHYDriver:78.3,Init cuphydriver:0.0,Make PHYDriverProxy:0.0,Init SCF FAPI:0.0,Create PHY_group:2.7,Start PHY_group:0.0, (total: 3229268.5us),

06:15:47.208410 CON phy_main 0 [CTL.STARTUP_TIMES] {TI DURATION} Start Main:10783.457,Parse Cuphycontroller YAML:1062.916,Init nvlog:12900.808,Cuda Set Device:560928.645,Cuphy PTI Init:27039.285,Init PHYDriver:2529606.398,Init cuphydriver:0.640,Make PHYDriverProxy:7.168,Init SCF FAPI:1.056,Create PHY_group:86736.791,Start PHY_group:201.345,

06:15:47.208412 CON phy_main 0 [CTL.STARTUP_TIMES] {TI TIMESTAMPS} Start Main:1762150543979121097,Parse Cuphycontroller YAML:1762150543989904554,Init nvlog:1762150543990967470,Cuda Set Device:1762150544003868278,Cuphy PTI Init:1762150544564796923,Init PHYDriver:1762150544591836208,Init cuphydriver:1762150547121442606,Make PHYDriverProxy:1762150547121443246,Init SCF FAPI:1762150547121450414,Create PHY_group:1762150547121451470,Start PHY_group:1762150547208188261,End Main:1762150547208389606,

06:15:47.208434 CON 144 0 [L2A.MODULE] cell_update_thread_func: OAM thread affinity set to cpu core 10
06:15:47.209129 CON msg_processing 0 [L2A.MODULE] Thread thread_func on CPU 9 initialized fmtlog
06:15:48.108358 CON UlPhyDriver05 0 [DRV.WORKER] Thread UlPhyDriver05(worker_default) on CPU 5 initialized fmtlog
06:15:48.108424 CON UlPhyDriver04 0 [DRV.WORKER] Thread UlPhyDriver04(worker_default) on CPU 4 initialized fmtlog
06:15:48.108462 CON DlPhyDriver08 0 [DRV.WORKER] Thread DlPhyDriver08(worker_default) on CPU 8 initialized fmtlog
06:15:48.108504 CON DlPhyDriver07 0 [DRV.WORKER] Thread DlPhyDriver07(worker_default) on CPU 7 initialized fmtlog
06:15:48.108513 CON DlPhyDriver06 0 [DRV.WORKER] Thread DlPhyDriver06(worker_default) on CPU 6 initialized fmtlog
06:16:05.278346 CON 142 0 [NVIPC.EFD] [nvipc] nvipc unix socket server connected
06:16:05.278386 CON 142 0 [NVIPC.EFD] [nvipc] Received peer event_fd: 228
06:16:05.478466 CON 142 0 [NVIPC.EFD] Share event_fd succeed: efd_tx=228, efd_rx=238
06:16:05.478467 CON 142 0 [NVIPC.EFD] [nvipc] Listening for nvipc client to connect …
06:16:06.385240 WRN msg_processing 0 [SCF.PHY] RECV: msg_len is not set correctly: cell_id=0 msg_id=0x02 msg_len=6201 head_len=8 body_len=6185
06:16:06.385253 CON msg_processing 0 [SCF.PHY] on_config_request: cell_id=0 numTLVs=33 state=0
06:16:06.385504 WRN msg_processing 0 [SCF.PHY] on_config_request unknown config request message ID 4171
06:16:06.385538 CON msg_processing 0 [SCF.PHY] PHY Cell Id = 51, M-Plane Id= 1
06:16:06.392504 CON msg_processing 0 [DRV.PUSCH] tvStatPrms: PUSCH enableDeviceGraphLaunch=1 enableCsiP2Fapiv3 = 0 nMaxLdpcHetConfigs = 32
06:16:06.392508 CON msg_processing 0 [DRV.PUSCH] static_params.nMaxPrb: 273
06:16:06.392508 CON msg_processing 0 [DRV.PUSCH] static_params.nMaxRx: 4
06:16:06.398331 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 54.706 MiB for cuPHY PUSCH channel object (0x52f4d4b6000).
06:16:06.398332 CON msg_processing 0 [CUPHY.PUSCH_RX] PuschRx: Running with eqCoeffAlgo 3
06:16:06.401978 CON msg_processing 0 [DRV.PUSCH] tvStatPrms: PUSCH enableDeviceGraphLaunch=1 enableCsiP2Fapiv3 = 0 nMaxLdpcHetConfigs = 32
06:16:06.401978 CON msg_processing 0 [DRV.PUSCH] static_params.nMaxPrb: 273
06:16:06.401978 CON msg_processing 0 [DRV.PUSCH] static_params.nMaxRx: 4
06:16:06.407007 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 54.706 MiB for cuPHY PUSCH channel object (0x52f4d4c2000).
06:16:06.407007 CON msg_processing 0 [CUPHY.PUSCH_RX] PuschRx: Running with eqCoeffAlgo 3
06:16:06.410611 CON msg_processing 0 [DRV.PUSCH] tvStatPrms: PUSCH enableDeviceGraphLaunch=1 enableCsiP2Fapiv3 = 0 nMaxLdpcHetConfigs = 32
06:16:06.410611 CON msg_processing 0 [DRV.PUSCH] static_params.nMaxPrb: 273
06:16:06.410612 CON msg_processing 0 [DRV.PUSCH] static_params.nMaxRx: 4
06:16:06.415784 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 54.706 MiB for cuPHY PUSCH channel object (0x52f4d4ce000).
06:16:06.415785 CON msg_processing 0 [CUPHY.PUSCH_RX] PuschRx: Running with eqCoeffAlgo 3
06:16:06.417966 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 3.341 MiB for cuPHY PUCCH channel object (0x52f4ec50000).
06:16:06.418947 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 3.341 MiB for cuPHY PUCCH channel object (0x52f4ec50e00).
06:16:06.419730 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 3.341 MiB for cuPHY PUCCH channel object (0x52f4ec51c00).
06:16:06.420569 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 3.341 MiB for cuPHY PUCCH channel object (0x52f4ec52a00).
06:16:06.422351 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.112 MiB for cuPHY PRACH channel object (0x52f4e6fd600).
06:16:06.422848 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.112 MiB for cuPHY PRACH channel object (0x52f4e6fd7c0).
06:16:06.442199 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 3.587 MiB for cuPHY SRS channel object (0x52f4e30d280).
06:16:06.459614 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 3.587 MiB for cuPHY SRS channel object (0x52f4e30d600).
06:16:06.476765 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 3.587 MiB for cuPHY SRS channel object (0x52f4e30d980).
06:16:06.482458 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f4ec31800).
06:16:06.487280 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f4ec39000).
06:16:06.491997 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f4ec3a800).
06:16:06.497076 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f4ec3c000).
06:16:06.502228 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f4ec3d800).
06:16:06.507222 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f4ec3f000).
06:16:06.512415 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f64840c00).
06:16:06.517511 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f64842400).
06:16:06.522806 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f64843c00).
06:16:06.527988 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 98.130 MiB for cuPHY PDSCH channel object (0x52f64845400).
06:16:06.529408 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30e080).
06:16:06.529617 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30dd00).
06:16:06.529762 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30e780).
06:16:06.529914 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30e400).
06:16:06.530065 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30eb00).
06:16:06.530216 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30ee80).
06:16:06.530367 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30f200).
06:16:06.530507 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30f580).
06:16:06.530641 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30f900).
06:16:06.530777 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.001 MiB for cuPHY SSB channel object (0x52f4e30fc80).
06:16:06.531056 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f4eb7e100).
06:16:06.531223 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f4eb7f000).
06:16:06.531469 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f4eb7f500).
06:16:06.531612 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f4eb7fa00).
06:16:06.531866 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f65b60000).
06:16:06.532010 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f65b60500).
06:16:06.532327 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f65b60a00).
06:16:06.532638 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f65b60f00).
06:16:06.532887 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f65b61400).
06:16:06.533044 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.965 MiB for cuPHY PDCCH channel object (0x52f65b61900).
06:16:06.533254 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658fbd80).
06:16:06.533389 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658fc080).
06:16:06.533623 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658fc380).
06:16:06.533746 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658fe180).
06:16:06.533846 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658fe480).
06:16:06.533949 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658fe780).
06:16:06.534049 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658fea80).
06:16:06.534146 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658fed80).
06:16:06.534254 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658ff080).
06:16:06.534555 CON msg_processing 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 0.093 MiB for cuPHY CSIRS channel object (0x52f658ff380).
06:16:06.534568 CON msg_processing 0 [DRV.API] Update cell: mplane_id=1 dl_grid_sz=273
06:16:06.534569 CON msg_processing 0 [DRV.API] Update cell: mplane_id=1 ul_grid_sz=273
06:16:06.534574 CON msg_processing 0 [SCF.PHY] on_config_request: create_cell_configs for cell_id=0 phy_cell_id=51 returned error_code=0
06:16:06.534585 CON msg_processing 0 [SCF.PHY] Send CONFIG.response: cell_id=0 error_code=0x0
06:16:06.534613 CON msg_processing 0 [L2A.TRANSPORT] wrapper: configured_cells_mask=0x1 expected_mask=0x1 all_cells_configured=true - all cells configured
06:16:06.534645 WRN msg_processing 0 [SCF.PHY] RECV: msg_len is not set correctly: cell_id=0 msg_id=0x04 msg_len=16 head_len=8 body_len=0
06:16:06.534647 CON msg_processing 0 [SCF.PHY] on_cell_start_request: Cell 0 Received START.request, sending START.response… phy_cell_id=51
06:16:06.534804 CON msg_processing 0 [L2A.TRANSPORT] wrapper: configured_cells_mask=0x1 expected_mask=0x1 all_cells_configured=true - all cells configured
06:16:06.536554 CON timer_thread 0 [L2A.TICK] Thread slot_indication_thread_sleep_method initialized fmtlog
06:16:06.536557 CON timer_thread 0 [L2A.TICK] PTP Configs: gps_alpha: 0 gps_beta: 0
06:16:06.536561 CON timer_thread 0 [L2A.TICK] Start time: tick=1762150566536555266, seconds since epoch = 1762150566, nanoseconds = 536555266
06:16:06.536562 CON timer_thread 0 [L2A.TICK] FIRST tick scheduled for: tick=1762150567480000000, seconds since epoch = 1762150567, nanoseconds = 480000000
06:16:07.480026 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.00 Mbps 0 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 0
06:16:07.511972 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 2.19 Slot Map 1 cell index 0 Dyn index -1 setting as unhealthy!
06:16:08.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.00 Mbps 0 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 2000
06:16:08.510414 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 102.19 Slot Map 101 cell index 0 Dyn index -1 setting as healthy!
06:16:08.531963 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 104.19 Slot Map 103 cell index 0 Dyn index -1 setting as unhealthy!
06:16:09.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 4000
06:16:09.530414 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 204.19 Slot Map 203 cell index 0 Dyn index -1 setting as healthy!
06:16:09.551964 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 206.19 Slot Map 205 cell index 0 Dyn index -1 setting as unhealthy!
06:16:10.480008 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 6000
06:16:10.550413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 306.19 Slot Map 305 cell index 0 Dyn index -1 setting as healthy!
06:16:10.571965 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 308.19 Slot Map 307 cell index 0 Dyn index -1 setting as unhealthy!
06:16:11.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 8000
06:16:11.570414 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 408.19 Slot Map 407 cell index 0 Dyn index -1 setting as healthy!
06:16:11.591962 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 410.19 Slot Map 409 cell index 0 Dyn index -1 setting as unhealthy!
06:16:12.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 10000
06:16:12.590415 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 510.19 Slot Map 509 cell index 0 Dyn index -1 setting as healthy!
06:16:12.611963 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 512.19 Slot Map 511 cell index 0 Dyn index -1 setting as unhealthy!
06:16:13.480007 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 12000
06:16:13.610413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 612.19 Slot Map 99 cell index 0 Dyn index -1 setting as healthy!
06:16:13.631965 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 614.19 Slot Map 101 cell index 0 Dyn index -1 setting as unhealthy!
06:16:14.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 14000
06:16:14.630413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 714.19 Slot Map 201 cell index 0 Dyn index -1 setting as healthy!
06:16:14.651965 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 716.19 Slot Map 203 cell index 0 Dyn index -1 setting as unhealthy!
06:16:15.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 16000
06:16:15.650413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 816.19 Slot Map 303 cell index 0 Dyn index -1 setting as healthy!
06:16:15.671966 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 818.19 Slot Map 305 cell index 0 Dyn index -1 setting as unhealthy!
06:16:16.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 18000
06:16:16.670413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 918.19 Slot Map 405 cell index 0 Dyn index -1 setting as healthy!
06:16:16.691964 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 920.19 Slot Map 407 cell index 0 Dyn index -1 setting as unhealthy!
06:16:17.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 20000
06:16:17.690415 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 1020.19 Slot Map 507 cell index 0 Dyn index -1 setting as healthy!
06:16:17.711966 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 1022.19 Slot Map 509 cell index 0 Dyn index -1 setting as unhealthy!
06:16:18.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 22000
06:16:18.710412 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 98.19 Slot Map 97 cell index 0 Dyn index -1 setting as healthy!
06:16:18.731963 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 100.19 Slot Map 99 cell index 0 Dyn index -1 setting as unhealthy!
06:16:19.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 24000
06:16:19.730413 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 200.19 Slot Map 199 cell index 0 Dyn index -1 setting as healthy!
06:16:19.751962 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 202.19 Slot Map 201 cell index 0 Dyn index -1 setting as unhealthy!
06:16:20.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 26000
06:16:20.750415 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 302.19 Slot Map 301 cell index 0 Dyn index -1 setting as healthy!
06:16:20.771961 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 304.19 Slot Map 303 cell index 0 Dyn index -1 setting as unhealthy!
06:16:21.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 28000
06:16:21.770414 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 404.19 Slot Map 403 cell index 0 Dyn index -1 setting as healthy!
06:16:21.791964 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 406.19 Slot Map 405 cell index 0 Dyn index -1 setting as unhealthy!
06:16:22.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 30000
06:16:22.790415 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 506.19 Slot Map 505 cell index 0 Dyn index -1 setting as healthy!
06:16:22.811965 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 508.19 Slot Map 507 cell index 0 Dyn index -1 setting as unhealthy!
06:16:23.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 32000
06:16:23.810413 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 608.19 Slot Map 95 cell index 0 Dyn index -1 setting as healthy!
06:16:23.831966 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 610.19 Slot Map 97 cell index 0 Dyn index -1 setting as unhealthy!
06:16:24.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 34000
06:16:24.830416 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 710.19 Slot Map 197 cell index 0 Dyn index -1 setting as healthy!
06:16:24.851963 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 712.19 Slot Map 199 cell index 0 Dyn index -1 setting as unhealthy!
06:16:25.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 36000
06:16:25.850413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 812.19 Slot Map 299 cell index 0 Dyn index -1 setting as healthy!
06:16:25.871966 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 814.19 Slot Map 301 cell index 0 Dyn index -1 setting as unhealthy!
06:16:26.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 38000
06:16:26.870414 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 914.19 Slot Map 401 cell index 0 Dyn index -1 setting as healthy!
06:16:26.891962 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 916.19 Slot Map 403 cell index 0 Dyn index -1 setting as unhealthy!
06:16:27.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 40000
06:16:27.890414 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 1016.19 Slot Map 503 cell index 0 Dyn index -1 setting as healthy!
06:16:27.911969 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 1018.19 Slot Map 505 cell index 0 Dyn index -1 setting as unhealthy!
06:16:28.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 42000
06:16:28.910413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 94.19 Slot Map 93 cell index 0 Dyn index -1 setting as healthy!
06:16:28.931966 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 96.19 Slot Map 95 cell index 0 Dyn index -1 setting as unhealthy!
06:16:29.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 44000
06:16:29.930414 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 196.19 Slot Map 195 cell index 0 Dyn index -1 setting as healthy!
06:16:29.951966 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 198.19 Slot Map 197 cell index 0 Dyn index -1 setting as unhealthy!
06:16:30.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 46000
06:16:30.950414 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 298.19 Slot Map 297 cell index 0 Dyn index -1 setting as healthy!
06:16:30.971965 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 300.19 Slot Map 299 cell index 0 Dyn index -1 setting as unhealthy!
06:16:31.480007 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 48000
06:16:31.970413 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 400.19 Slot Map 399 cell index 0 Dyn index -1 setting as healthy!
06:16:31.991965 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 402.19 Slot Map 401 cell index 0 Dyn index -1 setting as unhealthy!
06:16:32.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 50000
06:16:32.990414 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 502.19 Slot Map 501 cell index 0 Dyn index -1 setting as healthy!
06:16:33.011967 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 504.19 Slot Map 503 cell index 0 Dyn index -1 setting as unhealthy!
06:16:33.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 52000
06:16:34.010415 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 604.19 Slot Map 91 cell index 0 Dyn index -1 setting as healthy!
06:16:34.031965 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 606.19 Slot Map 93 cell index 0 Dyn index -1 setting as unhealthy!
06:16:34.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 54000
06:16:35.030414 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 706.19 Slot Map 193 cell index 0 Dyn index -1 setting as healthy!
06:16:35.051963 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 708.19 Slot Map 195 cell index 0 Dyn index -1 setting as unhealthy!
06:16:35.480006 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 56000
06:16:36.050412 ERR UlPhyDriver05 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 808.19 Slot Map 295 cell index 0 Dyn index -1 setting as healthy!
06:16:36.071967 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 810.19 Slot Map 297 cell index 0 Dyn index -1 setting as unhealthy!
06:16:36.480005 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.04 Mbps 50 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 58000
06:16:37.070413 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 910.19 Slot Map 397 cell index 0 Dyn index -1 setting as healthy!
06:16:37.091965 ERR UlPhyDriver04 0 [AERIAL_CUPHY_API_EVENT] [DRV.FUNC_UL] SFN 912.19 Slot Map 399 cell index 0 Dyn index -1 setting as unhealthy!
06:16:37.480004 CON timer_thread 0 [SCF.PHY] Cell 0 | DL 0.03 Mbps 41 Slots | UL 0.00 Mbps 0 Slots CRC 0 ( 0) | Tick 60000

furuki-takuma409 · November 4, 2025, 8:20am

I set pcp: 7 then cell being unhealthy disappeared. However DU and RU is not connected. When PCP is set to 7, Wireshark cannot see packets from RU, but when set to other numbers, Wireshark can see packets from RU instead, causing the unhealthy error to recur.

furuki-takuma409 · November 4, 2025, 9:33am

following is switch mirroring pcap.
It seems that ORU communicating with gNB server.

20251104_mirror_pcp7.pcap.zip (5.2 MB)

bkecicioglu · November 4, 2025, 9:17pm

@furuki-takuma409 If the cell unhealthy messages disappeared, the DU should be receiving UL packets.

Can you explain how did you conclude that the DU and the RU are not connected?

Thank you.

furuki-takuma409 · November 5, 2025, 12:36am

Thank you for your reply.

LiteOn RU shows “DU connection“ if DU and RU are connected. However my RU didn’t shows that. So I conclude the DU and the RU are not connected. O-RU status shows DU not connected but can I connect them and UE?

Thank you.

furuki-takuma409 · November 5, 2025, 6:16am

Now I think there is difference of mplane’s vlan id between RU and DU. I’m finding settings of DU’s that.
Where is the settings of DU’s m-plane-vlan id?

jixu · November 7, 2025, 3:25am

when DU and RU are not synced, you will also see ‘unhealthy’ message when pcp, vlan and MAC address, and FH NIC configurations are proper. Please check if DU and RU are synced.

In your case, the unhealthy disappeared while changing pcp to 7. it actually indicates pcp 7 is right configuration, so that there is no connection between DU and RU.

furuki-takuma409 · November 7, 2025, 8:10am

What should I do in that case?

The L2 switches and Grandmaster are configured according to the manual, and we have confirmed that the RU and DU are synchronized.

Following pcap seems DU and RU have connection on C-plane and U-plane but why is there no connection between DU and RU?

20251104_mirror_pcp7.pcap.zip (5.2 MB)

Topic		Replies	Views
Mlx5_common: Fail to create MR for address (0x7ee1d4000000) Aerial Forum	14	155	February 2, 2026
cuBB Termination During Interoperability Testing with Third-Party O-RU Aerial Forum aerial-cuda-accelerated-ran	10	153	December 17, 2025
[DRV.FUNC_UL] Slot Map 103, SFN 104 Slot 19 Order kernel timeout error or Exit error for cell index 0 Dyn index -1! Aerial Forum aerial-cuda-accelerated-ran	7	203	July 24, 2024
Running F08 cuPHYController+testmac+rusim fail in one system Aerial Forum aerial-cuda-accelerated-ran	12	107	November 7, 2025
Question about running cuPHYController Aerial Forum aerial-cuda-accelerated-ran	6	178	October 30, 2024
UL uplane cant received by cuphy (real RU) (rx_phy_bytes always=0) Aerial Forum aerial-cuda-accelerated-ran	8	135	January 8, 2026
Segmentation fault when running F08 E2E test (testMAC + cuPHY Controller + RU Emulator) Aerial Forum	1	50	January 30, 2026
Arieal SDK (23-3) setup: E2E test issue: RU data not received by DU server Container: CUDA cuda	0	184	May 15, 2024
Issue in running multi cell test Aerial Forum aerial-cuda-accelerated-ran	11	207	February 28, 2025
AERIAL ARC With Foxconn RPQN-7800E for N78 Aerial Forum aerial-ran-colab	44	439	November 4, 2024

Mlx5_common: Fail to create MR for address has been occurred

Related topics