Load failed for Torchscript model

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 5.0.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 7.0.0
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Trying to run a Deepsteream app with triton server and a Torchscript model(resnet_3d) taken from pytorch model repo.
Pytorch version: 1.6.0
model is successfully converted to Torchscript using torch.jit.trace but getting this error with Deepstream.
failed to load 'resnet_3d' version 1: Internal: load failed for libtorch model -> 'resnet_3d': version_ <= kMaxSupportedFileFormatVersion INTERNAL ASSERT FAILED at ../caffe2/serialize/inline_container.cc:132, please report a bug to PyTorch. Attempted to read a PyTorch file with version 3, but the maximum supported version for reading is 2. Your PyTorch installation may be too old.

Hi,

Based on the log, it seems you use a different PyTorch version to convert the model into Torchscript.
Since the installation depends on v1.6.0 v1.5.0, could you also generate the model with same version?

Thanks.

Hi @AastaLLL, I am using pytorch 1.6.0 to convert the model into torchscript.

Hi,

Sorry that some incorrect information in my previous comment.

Deepstream requires a v20.03 Triton server.
And based on the document below, v20.03 uses PyTorch 1.5.0a0+8f84ded.

https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html

The version difference leads to this error.
Is it possible to downgrade your PyTorch to v1.5.0?

Thanks.

Thanks @AastaLLL it worked with torch 1.5.0. just have a couple of queries.

  1. The above-mentioned doc shows Deepstream containers or triton ? since I am using a 20.09 container and it should support torch 1.7
  2. When can we expect to have a Deepstream container available with torch 1.6 support?

Hi,

Please find the below document that Deepstream requires a v20.03 Triton server.
That’s why it limits user for a v1.5.0-based model.
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_FAQ.html#triton

We are going to release a new Deepstream package, and it may support the newer Triton version.
Please pay attention to our announcement and update.

Thanks.