Model Parallelism Inference Across Multiple Devices

Autonomous Machines Jetson & Embedded Systems Jetson Projects
1

Hi,

What tools are recommended for doing Model Parallelism for Inference across multiple Jetson devices?

3

For which Jetson platform?

1 Like
4

Hi,

Could you share more information about your use case?

Do you want to infer the same input on different platforms for acceleration?
Or do you want to infer multiple inputs and each input run parallel on a device?

Thanks.

1 Like
5

For the Orin.

A single input for a model that has weights across devices.

The purpose of using multiple models is for acceleration

6

Hi,

We don’t have a library to separate the model into difference devices.
This might also be a model-dependent task.

Maybe others can share their experience here.
Thanks.