Isaac Sim Multi GPUs get slower proformance than one GPU

Omniverse Isaac Sim
, , , ,
25

Thanks, can you try one more thing. Can you compare the performance with 1x A6000 vs 1x RTX 5000? Try disabling the second GPU or pull it out if possible. Also, are you using NVLink?

I believe that one fast GPU will give you better FPS but adding more GPUs will help scale you workflows, allowing you to work on bigger scenes and more sensors/viewports at the same time.

26

Hi @vic-chen - did you get a chance to follow the instructions provided by Sheikh in the previous post?

27

Hi @Sheikh_Dawood ,
We don’t use NVlink. And following is compare the performance with RTX 5000 and RTX A6000.
RTX 5000
image
RTX A6000
image

28

Thanks. That perf difference looks about right.
Which usd file do you load for this test?

29

Hi @Sheikh_Dawood ,
I tested it by use our digit twins proejct. This scenarios is a real factory product line.
According to currently GPU performance, I think it can’t run our project.

30

Hi. How many object do you have in the scene? Does the FPS improve in a smaller scenario with less objects?

31

Hi @Sheikh_Dawood ,
My project have 1374 objects. If I use a smaller scenario, it has higher FPS.

32

Hi @Sheikh_Dawood ,
Because our hardware can’t run our project effectively, can you give some suggestions about hardware to choose from ?

33

@vic-chen can you press F8 and select “Gpu profiler” and send a screenshot
and then a second screenshot with F8 and “Cpu profiler” with the “CPU depth” set to at least 6.

Press “Pause updates” before taking the screenshot.

This will help us understand if the performance limitation is on the CPU side or the GPU.

34

Hi @Hammad_M ,
The following pictures are select Gpu Profiler.

image
image1920×1030 125 KB

image
image1920×1030 124 KB

image
image1920×1030 127 KB

And the following pitcures are select Cpu Profiler.
image
image1920×1030 131 KB

image
image1920×1030 126 KB

35

@vic-chen it looks the CPU is the primary bottleneck here.
Are most of the objects in the scene moving with physics?
Are you using the ROS bridge here?
Are your running any additional extensions that do CPU work?

Can you try opening/running your scene with
isaac-sim.fabric.sh instead of isaac-sim.sh

This uses a new backend for physics that greatly reduces the CPU overhead of updating USD. It also disables updates to USD so depending on how you are interacting with the scene, alternative APIs that avoid USD and its overhead could be used.

36

Hi @Hammad_M ,

Are most of the objects in the scene moving with physics?
Only 23/1374 objects have added physics attribute.
Are you using the ROS bridge here?
No.
Are your running any additional extensions that do CPU work?
I only added conveyor belt extensions, but I’m not sure if it runs with CPU.
Can you try opening/running your scene with
isaac-sim.fabric.sh instead of isaac-sim.sh
Yes, I have tried it. But it seems not to have improvement.