Hello everyone,
I am currently researching the development and deployment of AI and RAN (Radio Access Network) applications, aiming to explore how to effectively separate AI applications from RAN workloads to optimize performance.
Since AI applications often consume a significant portion of H2D bandwidth, we plan to leverage NVIDIA’s MIG (Multi-Instance GPU) technology to allocate RAN and AI workloads to different GPU instances, enabling efficient resource utilization and isolation.
However, during practical implementation, we encountered some issues:
After configuring according to the official NVIDIA MIG guide, we attempted to launch Docker containers related to ARC-OTA but found that the containers failed to start.
The ARC-OTA configuration file is attached.
The logs do not provide clear error messages, but it appears to be related to MIG instance binding issues.
Here are some questions I would like to ask:
Are there best practices for MIG configuration in ARC-OTA scenarios?
Are there specific guidelines for allocating RAN and AI workloads to MIG instances?
Are there successful cases or debugging tips for similar scenarios that you can share?
System Environment Information:
GPU Configuration:
GPU Model: NVIDIA A100XDriver Version: 555.42.02CUDA Version: 12.5MIG Configuration:
GPU 0: MIG mode enabled, with two MIG instances
MIG 1: 38MiB / 40192MiB
MIG 2: 50MiB / 40192MiB
GPU 1: MIG mode not enabled
CPU Configuration:
CPU Model: Intel(R) Xeon(R) Gold 6336Y CPU @ 2.40GHzCore Count: 48 (24 cores per socket, 2 sockets total)NUMA Nodes: 2 (NUMA Node 0: even cores, NUMA Node 1: odd cores)
Other Configuration Information:
Docker Runtime: NVIDIA Container RuntimeARC-OTA Version: 1.5AI Application Framework and Version: vLLM llava-v1.6-mistral-7b-hf
Thank you for your assistance! Any suggestions or resources would greatly benefit my current research and development work.
Thanks again, and I look forward to your response!
sit.zip (11.1 KB)
Additional Information:https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
Hi and welcome
$ nvidia-smi -L
And in the docker compose file add in the cubb section:
environment:
- NVIDIA_VISIBLE_DEVICES=MIG-1dba3a4b-52e5-5d46-b9f0-84a1e21d0faf
And also depending on the MIG setup you have to change the number of SMs in the cuphycontrol.yaml file.
mps_sm_pusch: 82
Thank you for your response and helpful suggestions!
However, I am currently using the NVIDIA A100X GPU, configured in MIG mode with two instances:
MIG 1: 38MiB / 40192MiBMIG 2: 50MiB / 40192MiB
I would like to know if there are any recommended Docker Compose configuration parameters and SM settings in the cuphycontrol.yaml file specifically for the A100X. Particularly, I’m looking for advice on efficiently allocating RAN and AI workloads. Are there any specific guidelines or reference configurations for this scenario?
Thank you again for your help, and I look forward to your guidance!
jixu
January 10, 2025, 11:32pm
6
For the case you parted the GPU of A100X, assume the partition is 3g.40gb and 4g.40gb. The corresponding SM number of each MIG device should be SM=42 and 56 for 3g.40gb and 4g.40gb, repectively.
example MIG device ID:
nvidia-smi -L
GPU 0: NVIDIA A100X (UUID: GPU-aad78ddf-0187-6c32-a882-b3b50bd46976)
MIG 4g.40gb Device 0: (UUID: MIG-4292c581-87da-5569-8483-1b43dff9795b)
MIG 3g.40gb Device 1: (UUID: MIG-ac313f7c-2ea5-5f74-8f07-93ab05ea6bf0)
In the Docker compose, replace the environment NVIDIA_VISIBLE_DEVICES with the UUID of the MIG device you choose to use. For example, choose mig-3g.40gb for RAN
NVIDIA_VISIBLE_DEVICES=MIG-1dba3a4b-52e5-5d46-b9f0-84a1e21d0faf
in cuphycontroller_P5G_FXN.yaml, scale down the mps_sm_* values accordingly based on the available SM (ex: SM= 42 for mig-3g.48gb). A reference configuration as shown below,
1 Like
system
March 10, 2025, 7:30am
7
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
