TensorRT, Python and Windows

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I have a Tensorflow model trained in Python on a Windows machine.
I plan to convert to UFF and do inference to optimize “execution time”.

I can read in other posts, that for Python samples and UFF converter, then install DEB package.
Does that mean TensorRT inference and the following execution/deployment can only be done from Linux?

Or how to install the UFF parser and TensorRT for Python in a Windows environment?

2

Hello,

The UFF file is platform independent, so you can use your UFF on linux.

Windows installation of TensorRT includes a collection of Parsers, which inlcudes the UFF parser. It’s not a separate installation.

3

But according to documentation it seems like no Python support for TensorRT on Windows…?

So to achieve deployment on TensorRT engine for a Tensorflow model, either:

  1. go via C++ API on Windows, and do UFF conversion and TensorRT inference in C++.

Or

  1. if I prefer Python, I must change to Linux OS, and then it is possible to use UFF converter and TensorRt inference via Python on Linux.

Correct?

1 Like
4

Hello,

Yes. if you must use Python, you’d use Linux.

Apologies for the inconvenience.

5

Hello,

Do you plan to allow the installation of the Python API on Windows?

Thank you.

1 Like
6

How is it that 1.5 years later there still isn’t Python support for TensorRT on Windows?

4 Likes
7

Here is a Windows compatible lib: GitHub - KorovkoAlexander/tensorrt_models: Project aimed at making cpp inference on TensorRT from python

1 Like
8

May I ask if you have used this library? In my test, I found that the pure infer time of this library was much slower than that of TensorRT C++, and even much slower than that of the original pytorch model. However, the predicted result was correct, which made me very confused.

9

i need too

1 Like
10

in this thread :

author said that he is using python tensorrt on win10 ? how is that possible?

11

sorry, this is the link
https://github.com/NVIDIA/TensorRT/issues/203

12

Hello,
From TRT8 Installation guide TRT Install Getting Started

It mentioned:

  • The Windows zip package for TensorRT does not provide Python support. Python may be supported in the future.

Can NVIDIA provide please an estimated date\release when there will be a TRT Python API supported on Windows?

Thanks,

1 Like
13

Hi , UFF and Caffe Parser have been deprecated from TensorRT 7 onwards, hence request you to try ONNX parser.
Please check the below link for the same.

Thanks!

16

Maybe I missed it but I cannot understand how last answer related to the question about TRT Python API readiness for Windows.

I will much appreciate if you will clarify it.

Thanks,