Pytorch to tensorRT conversion

Though I could able to migrate the model to TensorRT using torch2trt, I required the model to work with multiple sizes of images. The migrated model works with a single image size. How can I input dynamic sizes during the migration process?

I have got this link, but I am not sure how can we add dynamic properties while trying to migrate CNN model

Hi @karunakar.r ,

You can may be try an unofficial but working repository on torch2trt: GitHub - grimoire/torch2trt_dynamic: A pytorch to tensorrt convert with dynamic shape support

Let me know if your conversion is successful with this.

There is another latest support: GitHub - NVIDIA/TRTorch: PyTorch/TorchScript compiler for NVIDIA GPUs using TensorRT which can be explored as well.

Thanks @bgiddwani . When I try to run migration, I am getting below error
“OSError: /documents/trt/torch2trt_dynamic/torch2trt_dynamic/plugins/libamirstan_plugin.so: cannot open shared object file: No such file or directory”

I followed below steps to install

git clone https://github.com/grimoire/torch2trt_dynamic.git torch2trt_dynamic
cd torch2trt_dynamic
python3 setup.py develop

I even have tried installing plugins using below command
python3 setup.py install --plugins
this is giving error as “error: [Errno 2] No such file or directory: ‘ninja’: ‘ninja’”

Could you please guide us?

It is true sir. The plugin you mentioned is missing. To get it working we need to manually download the plugin and place it into the plugins folder.

Thanks @srimanthtenneti

I got stuck when try providing TensorRT path. I can see that TensorRT is available on the below location.
/usr/src/tensorrt/
However, as i am trying to run inside Docker, how can i provide TensorRT path?
GH location instruction for installing the plugin

If i install new TensorRT, would there be any impact on inference?

Hi @srimanthtenneti, I followed the instructions mentioned to install plugin. I am still getting the same error “OSError: /documents/trt/torch2trt_dynamic/torch2trt_dynamic/plugins/libamirstan_plugin.so: cannot open shared object file: No such file or directory”
Do I required to copy any folder from amirstan_plugin folder to torch2trt_dynamic folder

The below command gave error

amirstan_plugin/build# cmake -DTENSORRT_DIR=‘/usr/src/tensorrt’
CMake Error: The source directory “/documents/trt/amirstan_plugin/build” does not appear to contain CMakeLists.txt.

than, i moved to parent folder, and ran the same command.
[edit] I fixed above error after passing … after command. However, when I try to run
make -j10
in build folder, I am getting below error
make: *** No targets specified and no makefile found. Stop.

Do you want me to run this in parent folder?

@srimanthtenneti I have followed all the mentioned steps from below both GH links

While running migration script, I start getting below error. Can you please check
OSError: /documents/trt/torch2trt_dynamic/torch2trt_dynamic/plugins/libamirstan_plugin.so: cannot open shared object file: No such file or directory

I also reset the bash to reflect the changed configuration. When I run below command, its giving correct results

echo “$AMIRSTAN_LIBRARY_PATH”

@bgiddwani

The issues is .so file not automatically gets referenced from torch2trt_dynamic folder. I have copied the file manually to make it work

cp libamirstan_plugin.so /documents/trt/torch2trt_dynamic/torch2trt_dynamic/plugins/