Deploying Pytorch converted to ONNX

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi All,

I follow the article https://hk.saowen.com/a/557dc366e59daaff19bd37641351c6d14fd518a9e00af84026bfc9366a3c41e7 to deploy my own model on TensorRT container. I perform these steps:

a. Inspired by the the Inception V1 network trained using the ImageNet dataset in this article, I convert my Pytorch model’s weight to ONNX by getting the .pb and .txt files.
b. Modify accordingly the tensorrt_server executable.
For the prompt in the article “Make sure you set INPUT_NAME , INPUT_FORMAT and OUTPUT_NAME appropriately for your model”, I am not sure how to perform and I go with the default.
c. Bash inot the container, I try to run like this:

workspace/tensorrt_server# bash ./tensorflow_mymodel

, unfortunately with errors.

  • Without getting into details about specific errors, my questions is, am I missing any fundamental part to develop a deployable version of my model?
  • Is there any specific tutorial that will lead me step by step through this?

Thanks.

2

Hello,

I assume you are using the TensorRT Server Container? nvcr.io/nvidia/tensorrtserver:18.10-py3

Please reference example listed here: https://github.com/NVIDIA/dl-inference-server

3

Thanks. I 've seen this up-to-date TensorRT inference server, but I haven’t tried it yet.
My motivation is to understand the issues I got once I follow the (outdated) article above.

4

Hello,

To help us debug, can you describe the exact issue/error you are seeing? A small repro package containing the source/models that exhibit the symptoms will really help.

regards,
NVIDIA Enterprise Support