"How to connect/Combine Two Jetson AGX Orin Devices to Share GPU Workload?"

[parchuretejas]

Hello everyone,
I am working on a project where I have two Jetson AGX Orin devices, and I want to combine them in such a way that they can share the GPU workload for computational tasks. Specifically, I would like to use both devices together to pool their GPU resources, allowing for more efficient processing of demanding workloads, such as AI inference or heavy computational tasks.

Is it possible to use both Jetson AGX Orin devices to share GPU resources directly for a single application or task? I understand that the devices have separate GPUs, but is there any way to have them work together to balance the GPU load?

[kayccc]

We don’t have the reference design for cluster, also no benchmark/test/apps to to run on multiple Jetson devices in parallel.

[parchuretejas]

Okay , is there a way i can connect jetson agx orin with a nano for this purpose?

[kayccc]

Don’t know that, may someone help to share experiences.

[whitesscott]

I no longer have an orin nano to confirm this works, but 6 months back I believe the following worked for me as proof of concept test. I used python.

Create hosts file in your project directory, entering your true device IP addresses
192.168.1.10 slots=1 # IP of Jetson AGX Orin
192.168.1.11 slots=1 # IP of Jetson Orin Nano

on both Orins install:
sudo apt-get install -y libopenmpi-dev openmpi-bin
pip3 install cupy-cuda12x mpi4py

Set up passwordless SSH between the devices.
Try the test to have mpirun can run using both gpus.

mpirun -np 2 -hostfile hosts python3 distributed_gpu_task.py

distributed_gpu_task.py.txt (3.9 KB)

There may be other ways to do this.