Hello, Thanks for the response. It says “Device IDs of GPU to use for pre-processing/inference (single GPU support only)”. Does that mean nvinferserver can run inference either on GPU-0 or GPU-1. It can not do load balancing the way Triton is doing. Is that correct understanding?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Nvinferserver | 13 | 1520 | October 12, 2021 | |
| Triton Server GPU memory copy? | 10 | 1279 | May 10, 2023 | |
| How to utilize multiple GPU in deepstream 5.0 | 7 | 2653 | October 12, 2021 | |
| Deepstream with standalone triton server | 4 | 919 | November 1, 2021 | |
| Whether the nvinfer can work in parallel or not? | 2 | 1034 | October 2, 2021 | |
| Is there a way to balance the load for Nvinferserver when inferencing with intervals | 3 | 151 | June 18, 2024 | |
| Multi GPU set up | 7 | 1052 | August 23, 2022 | |
| How to add triton server to deepstream in different device? | 10 | 856 | September 7, 2023 | |
| Performance about nvinfer and nvinferserver | 6 | 1468 | March 22, 2022 | |
| Triton-server model load balancing | 6 | 998 | February 8, 2023 |