Does Python backend support KIND_GPU in tritonserver

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
, , ,
1

My device is jetson orin nano 16gb and i want to run tritonserver in my device. But i cannot start tritonserver because error : GPU instances not supported. Other models is ready except two models that have python backend model

image
image1825×512 74.5 KB

When i try to change instancegroup in file config.pbtxt of 2 unready model above from KIND_GPU to KIND_CPU, it works. Therefore, can anyone help me to solve this problem, what does i need to do to load all model with using GPU successfully. Thank you very much

3

Hi,

The Python backend does not support GPU Tensors and Async BLS

Are GPU Tensors used in your python models?
If not, could you share the model for us to reproduce?

Thanks.

4

Can you tell me how to check if GPU tensors used in python models. Thanks

5

Hi,

Please check this configuration:

Could you try to set the FORCE_CPU_ONLY_INPUT_TENSORS to be yes?
Thanks.

6

@AastaLLL I have try to set the FORCE_CPU_ONLY_INPUT_TENSORS to be both yes and no in the model configuration of two models align and preprocessing_without_landmark and instancegroup is KIND_GPU

image
image1707×507 85.8 KB

But it stills got the same error
image
image714×458 50.3 KB

This is the image of excecute function in file model.py of model align
image
image1037×772 114 KB

Can you help me to solve this ? Thanks

7

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi,

We want to reproduce this issue internally to check the root cause further.
Could you share the model and your steps in detail with us?

Thanks.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.