Query about why there's this MAX_N_TBS_PER_CELL_GROUP_SUPPORTED limit

Accelerated Computing Aerial Forum
1

Hi Aerial team,
we’re trying to run testVectors/TVnr_7891_PUSCH_gNB_CUPHY_s0p63.h5 with multiple cells in parallel on GH200 using the following command:
./build.aarch64/cuPHY/examples/pusch_rx_multi_pipe/cuphy_ex_pusch_rx_multi_pipe -i testcases/pusch_7891_4cells.yaml -m 1 -c 4 -r 10 -w 0

We ran into MAX_N_TBS_PER_CELL_GROUP_SUPPORTEDPUSCH this limit so it cannot run. Any reason why there’s this limit?
Input vector: /opt/nvidia/cuBB/testVectors/TVnr_7891_PUSCH_gNB_CUPHY_s0p63.h5 cells: 4 (/tmp/pusch_multicell_rnnp8jtt/TVnr_7891_PUSCH_gNB_CUPHY_s0p63_4cells.yaml)02:47:55.682258 CON 4549 0 [NVLOG.CPP] Using /opt/nvidia/cuBB/cuPHY/nvlog/config/nvlog_config.yaml for nvlog configuration02:47:55.686039 CON 4556 0 [NVLOG.EXIT_HANDLER] [exit_watchdog] thread started on core 002:47:55.691129 CON 4549 0 [NVLOG.CPP] Output log file path /tmp/pusch.logFile extension: yamlnumber of cells: 4slot: 0 channel: PUSCH file: /opt/nvidia/cuBB/testVectors/TVnr_7891_PUSCH_gNB_CUPHY_s0p63.h5 file: /opt/nvidia/cuBB/testVectors/TVnr_7891_PUSCH_gNB_CUPHY_s0p63.h5 file: /opt/nvidia/cuBB/testVectors/TVnr_7891_PUSCH_gNB_CUPHY_s0p63.h5 file: /opt/nvidia/cuBB/testVectors/TVnr_7891_PUSCH_gNB_CUPHY_s0p63.h5PuschRx Pipeline[0]: pid 4560 policy 2 prio 99PuschRx Pipeline[0]: pid 4560 set affinity to CPU 0PuschRx Pipeline[0]: Wait for start-sync point02:47:56.222547 CON 4560 0 [CUPHY.PUSCH_RX] LDPC throughput mode enabled02:47:56.245831 CON 4560 0 [CUPHY.MEMFOOT] cuphyMemoryFootprint - GPU allocation: 2587.097 MiB for cuPHY PUSCH channel object (0xf48a14058210).02:47:56.245832 CON 4560 0 [CUPHY.PUSCH_RX] PuschRx: Running with eqCoeffAlgo 3Run config: GPU Id 0, # of pipelines 1PuschRx Pipeline[0]: start-syncpoint hitPuschRx Pipeline[0]: For transmission 1 using filename /opt/nvidia/cuBB/testVectors/TVnr_7891_PUSCH_gNB_CUPHY_s0p63.h502:47:56.341741 ERR 4560 0 [AERIAL_CUPHY_EVENT] [CUPHY.PUSCH_RX] EXCEPTION: nTBs in TV > MAX_N_TBS_PER_CELL_GROUP_SUPPORTEDPUSCH example failed for /opt/nvidia/cuBB/testVectors/TVnr_7891_PUSCH_gNB_CUPHY_s0p63.h5 x4 cells (exit 1)

2

Hi @10910905 ,

Welcome to the forum!

We design cuPHY to pre-secure GPU memory when the initialization to meet the 5G latency requirement. For that reason, it is not allowed to secure the memory dynamically on the fly.
Furthermore, we employ CUDA graphs for the latency optimization. CUDA graphs need to fix the kernel arguments and buffer addresses when instantiating the graph. Therefore, it is not easy to change the buffer size in run-time.

Thank you,
Nori