Upload pytorch model failed

I convert my model to torchscript with torch 1.4.0 using the following code.

pt_name='seg_mr_3d_LPS_intervertebral_disc.pt'
img=img.to('cuda')
net=net.to('cuda')
traced_script_module=torch.jit.trace(net,img)
traced_script_module.save(pt_name)

And put my pt and config.json with code

curl -X PUT "http://0.0.0.0:5000/admin/model/seg_mr_3d_LPS_intervertebral_disc" \
-F "config=@seg_mr_3d_LPS_intervertebral_disc.json;type=application/json" \
-F "data=@seg_mr_3d_LPS_intervertebral_disc.pt"

But it’s failed and print like this.

Model/Config Not Found OR Failed to import model into AIAA
Load with valid model+config

I’m sure i can load the pt file and use it in Pytorch succussfully, the result is right.

model=torch.jit.load(pt_name)
output_pt=model(img)

The log in http://10.6.208.121:5000/logs end with:

ERROR](nvmidl.apps.aas.actions.model_import_trtis:import_model) - Gen TRTIS model failed: Unable to get status for 'seg_mr_3d_LPS_intervertebral_disc'

I switch to othor methods and also failed.

  1. pytorch 1.1.0, failed the same.
  2. Trace without cuda, failed the same.
img=np.random.rand(1,1,320,320,80)
img=img.astype(np.float32)
traced_script_module=torch.jit.trace(net,img)
traced_script_module.save(pt_name)
  1. I use libtorch in /opt/tensorrtserver/ of clara to load my pt file, it’s ok !
#include <torch/script.h>
#include <iostream>
#include <memory>

int main(int argc,const char* argv[]){
        if(argc !=2){
                std::cerr<<"usage: torch_test pt_name"<<std::endl;
                return -1;
        }
        torch::jit::script::Module module;
        module=torch::jit::load(argv[1]);
        std::cout<<"load ok"<<std::endl;
        return 1;
}

g++ -Wall -o torch_test torch_test.cpp -ltorch -lc10 -L/opt/tensorrtserver/lib -I/opt/tensorrtserver/include/torch -Wl,-R /opt/tensorrtserver/lib

./torch_test seg_mr_3d_LPS_intervertebral_disc.pt

I look forward to your answers, thank you very much

seg_mr_3d_LPS_intervertebral_disc.json

{
  "version": "2",
  "type": "segmentation",
  "labels": [
    "intervertebral disc"
  ],
  "description": "3D mr segmentation of intervertebral pretrained model based on vnet, code:10.6.208.121:/home/ia/user/JFG/model_org/IVD/vnet/",
  "pre_transforms": [
    {
      "name": "LoadNifti",
      "args": {
        "fields": "image"
      }
    },
    {
      "name": "ConvertToChannelsFirst",
      "args": {
        "fields": "image"
      }
    },
    {
      "name": "ScaleByResolution",
      "args": {
        "fields": "image",
        "target_resolution": [
          1.0,
          1.0,
          1.0
        ]
      }
    },
    {
      "name": "NormalizeNonzeroIntensities",
      "args": {
        "fields": "image"
      }
    }
  ],
  "inference": {
    "image": "image",
    "scanning_window": true,
    "batch_size": 8,
    "roi": [
      96,
      96,
      64
    ],
    "trtis": {
      "input_channels": {
        "image": 1
      },
      "output_channels": {
        "model": 2
      },
      "gpu_instance_count": 1,
      "max_batch_size": 8,
      "platform": "pytorch_libtorch"
    }
  },
  "post_transforms": [
    {
      "name": "ArgmaxAcrossChannels",
      "args": {
        "fields": "model"
      }
    },
    {
      "name": "FetchExtremePoints",
      "args": {
        "image_field": "image",
        "label_field": "model",
        "points": "points"
      }
    },
    {
      "name": "CopyProperties",
      "args": {
        "fields": [
          "model"
        ],
        "from_field": "image",
        "properties": [
          "affine"
        ]
      }
    },
    {
      "name": "RestoreOriginalShape",
      "args": {
        "field": "model",
        "src_field": "image",
        "is_label": true
      }
    }
  ],
  "writer": {
    "name": "WriteNifti",
    "args": {
      "field": "model",
      "dtype": "uint8"
    }
  }
}

Detailed Log Info.

[2020-03-11 06:56:31.058][ INFO](nvmidl.apps.aas.server:admin_model_load) - Model: seg_mr_3d_LPS_intervertebral_disc
[2020-03-11 06:56:31.058][ INFO](nvmidl.apps.aas.server:check_for_access) - Request Host: 0.0.0.0:5000
[2020-03-11 06:56:31.059][ INFO](nvmidl.apps.aas.server:check_for_access) - Remote  Addr: 127.0.0.1
[2020-03-11 06:56:32.141][ INFO](nvmidl.apps.aas.server:admin_model_load) - Reading Config from Input Files
[2020-03-11 06:56:32.141][ INFO](nvmidl.apps.aas.server:admin_model_load) - Input Model Config: 
{'version': '2', 'type': 'segmentation', 'labels': ['intervertebral disc'], 'description': '3D mr segmentation of intervertebral pretrained model based on vnet, code:10.6.208.121:/home/ia/user/JFG/model_org/IVD/vnet/', 'pre_transforms': [{'name': 'LoadNifti', 'args': {'fields': 'image'}}, {'name': 'ConvertToChannelsFirst', 'args': {'fields': 'image'}}, {'name': 'ScaleByResolution', 'args': {'fields': 'image', 'target_resolution': [1.0, 1.0, 1.0]}}, {'name': 'NormalizeNonzeroIntensities', 'args': {'fields': 'image'}}], 'inference': {'image': 'image', 'scanning_window': True, 'batch_size': 8, 'roi': [96, 96, 64], 'trtis': {'input_channels': {'image': 1}, 'output_channels': {'model': 2}, 'gpu_instance_count': 1, 'max_batch_size': 8, 'platform': 'pytorch_libtorch'}}, 'post_transforms': [{'name': 'ArgmaxAcrossChannels', 'args': {'fields': 'model'}}, {'name': 'FetchExtremePoints', 'args': {'image_field': 'image', 'label_field': 'model', 'points': 'points'}}, {'name': 'CopyProperties', 'args': {'fields': ['model'], 'from_field': 'image', 'properties': ['affine']}}, {'name': 'RestoreOriginalShape', 'args': {'field': 'model', 'src_field': 'image', 'is_label': True}}], 'writer': {'name': 'WriteNifti', 'args': {'field': 'model', 'dtype': 'uint8'}}}
[2020-03-11 06:56:32.203][ INFO](nvmidl.apps.aas.server:admin_model_load) - Saved archive file as: /workspace/data/aiaa/downloads/host-5000/seg_mr_3d_LPS_intervertebral_disc.pt
[2020-03-11 06:56:32.203][ INFO](nvmidl.apps.aas.actions.model_import:model_import_local) - Model: seg_mr_3d_LPS_intervertebral_disc
[2020-03-11 06:56:32.213][ INFO](nvmidl.apps.aas.actions.model_import:model_import_local) - Unpack Archive: /workspace/data/aiaa/downloads/host-5000/seg_mr_3d_LPS_intervertebral_disc.pt
[2020-03-11 06:56:32.213][ INFO](nvmidl.apps.aas.actions.model_import_trtis:_import_model_trtis) - generating trtis model dir on /workspace/data/aiaa/trtis_models with model seg_mr_3d_LPS_intervertebral_disc
[2020-03-11 06:58:33.880][ INFO](schedule:run) - Running job Every 5 minutes do cleanup_docs() (last run: 2020-03-11 06:53:33, next run: 2020-03-11 06:58:33)
[2020-03-11 07:01:44.578][ INFO](werkzeug:_log) - 10.8.112.2 - - [11/Mar/2020 07:01:44] "e[37mGET /logs HTTP/1.1e[0m" 200 -
[2020-03-11 07:01:44.629][ INFO](werkzeug:_log) - 10.8.112.2 - - [11/Mar/2020 07:01:44] "e[37mGET /favicon.ico HTTP/1.1e[0m" 200 -
[2020-03-11 07:02:34.667][ERROR](nvmidl.apps.aas.actions.model_import_trtis:import_model) - Gen TRTIS model failed: Unable to get status for 'seg_mr_3d_LPS_intervertebral_disc'

Hi,

Thanks for the information.

  1. Are you using “docker pull nvcr.io/nvidia/pytorch:19.09-py3”? this is the PyTorch container that our current Clara 2.0 is based on.
    Inside that container run the convert script you have.

  2. From your code, I saw the input dimension is (1, 1, 320, 320, 80). but in your config_aiaa.json, you specify ROI as 96, 96, 64, this would
    cause the input shape to be (1, 1, 96,96, 64). If your input image is the same size as your network input, please specify
    “scanning_window”: false and “roi”: [320, 320, 80]

Hope this helps