Deploying Pytorch converted to ONNX

Hi All,

I follow the article to deploy my own model on TensorRT container. I perform these steps:

a. Inspired by the the Inception V1 network trained using the ImageNet dataset in this article, I convert my Pytorch model’s weight to ONNX by getting the .pb and .txt files.
b. Modify accordingly the tensorrt_server executable.
For the prompt in the article “Make sure you set INPUT_NAME , INPUT_FORMAT and OUTPUT_NAME appropriately for your model”, I am not sure how to perform and I go with the default.
c. Bash inot the container, I try to run like this:

workspace/tensorrt_server# bash ./tensorflow_mymodel

, unfortunately with errors.

  • Without getting into details about specific errors, my questions is, am I missing any fundamental part to develop a deployable version of my model?
  • Is there any specific tutorial that will lead me step by step through this?



I assume you are using the TensorRT Server Container?

Please reference example listed here:

Thanks. I 've seen this up-to-date TensorRT inference server, but I haven’t tried it yet.
My motivation is to understand the issues I got once I follow the (outdated) article above.


To help us debug, can you describe the exact issue/error you are seeing? A small repro package containing the source/models that exhibit the symptoms will really help.

NVIDIA Enterprise Support