Hi,
Which frameworks do you use for inference? For python, please make sure you have installed the package that has built with the CUDA support:
Thanks.