Assertion Failed - inputs.at(1).is_weights()

Description

Hello all, I have converted the fasterrcnn model from pytorch to onnx and loading the onnx model from the c++ API and the following error is shown - ERROR: builtin_op_importers.cpp:2179 In function importPad: [8] Assertion failed: inputs.at(1).is_weights() Error could not parse.
How could Make this error go away, and this really an error? because the program does not stop and finishes with exit code 0.
TIA

Environment

TensorRT Version-7.1.3:
GPU Type-RTX 2070:
Nvidia Driver Version-440.100:
CUDA Version-10.2:
CUDNN Version-8.0.2:
Operating System + Version- Ubuntu 20.04LTS:
Python Version (if applicable) 3.8:
PyTorch Version (if applicable)-1.6:
Baremetal or Container (if container which image + tag)

Steps To Reproduce

Created Onnx model using -

from torchvision.models.detection import fasterrcnn_resnet50_fpn
import torch
import torch.onnx

model = fasterrcnn_resnet50_fpn(pretrained=True).eval()
a = torch.randn((1, 3, 640, 480))
out = model(a)
print(out)
torch.onnx.export(model, a, "fasterrcnn.onnx", export_params=True, opset_version=11)

This results in multiple warning which are as follows -


/home/atharva/.local/lib/python3.8/site-packages/torch/tensor.py:452: RuntimeWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  warnings.warn('Iterating over a tensor might cause the trace to be incorrect. '
/home/atharva/.local/lib/python3.8/site-packages/torch/nn/functional.py:3009: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  return [(torch.floor((input.size(i + 2).float() * torch.tensor(scale_factors[i],
/home/atharva/.local/lib/python3.8/site-packages/torchvision/models/detection/rpn.py:163: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  strides = [[torch.tensor(image_size[0] // g[0], dtype=torch.int64, device=device),
/home/atharva/.local/lib/python3.8/site-packages/torchvision/models/detection/rpn.py:164: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  torch.tensor(image_size[1] // g[1], dtype=torch.int64, device=device)] for g in grid_sizes]
/home/atharva/.local/lib/python3.8/site-packages/torchvision/ops/boxes.py:124: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  boxes_x = torch.min(boxes_x, torch.tensor(width, dtype=boxes.dtype, device=boxes.device))
/home/atharva/.local/lib/python3.8/site-packages/torchvision/ops/boxes.py:126: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  boxes_y = torch.min(boxes_y, torch.tensor(height, dtype=boxes.dtype, device=boxes.device))
/home/atharva/.local/lib/python3.8/site-packages/torchvision/models/detection/transform.py:269: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  torch.tensor(s, dtype=torch.float32, device=boxes.device) /
/home/atharva/.local/lib/python3.8/site-packages/torchvision/models/detection/transform.py:270: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  torch.tensor(s_orig, dtype=torch.float32, device=boxes.device)
/home/atharva/.local/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:2221: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  warnings.warn("Exporting aten::index operator of advanced indexing in opset " +

The c++ code used to load the model is -


class LoggerL : public nvinfer1::ILogger
{
public:
    void log(Severity severity, const char* msg) override {
        // remove this 'if' if you need more logged info
        if ((severity == Severity::kERROR) || (severity == Severity::kINTERNAL_ERROR)) {
            std::cout << msg << "n";
        }
    }
}gLogger;

struct TRTDestroy
{
    template<class T>
        void operator()(T* obj) const
        {
            if(obj)
                obj->destroy();
        }
};

template<class T>
using TRTUniquePtr = std::unique_ptr<T, TRTDestroy>;

void parseONNX(const std::string path,
               TRTUniquePtr<nvinfer1::ICudaEngine>& engine,
               TRTUniquePtr<nvinfer1::IExecutionContext> &context)
               {
    const auto explicitBatch = 1U << static_cast<uint32_t>(nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH);
    TRTUniquePtr<nvinfer1::IBuilder> builder{nvinfer1::createInferBuilder(gLogger)};
    TRTUniquePtr<nvinfer1::INetworkDefinition> network{builder->createNetworkV2(explicitBatch)};
    TRTUniquePtr<nvonnxparser::IParser> parser{nvonnxparser::createParser(*network, gLogger)};
    if (!parser->parseFromFile(path.c_str(), static_cast<int>(nvinfer1::ILogger::Severity::kINFO)))
    {
        std::cerr<<"Error could not parse";
        return;
    }
    TRTUniquePtr<nvinfer1::IBuilderConfig> config{builder->createBuilderConfig()};
    config->setMaxWorkspaceSize(1ULL<<30);
    if (builder->platformHasFastFp16())
        config->setFlag(nvinfer1::BuilderFlag::kFP16);
    builder->setMaxBatchSize(1);
    engine.reset(builder->buildEngineWithConfig(*network, *config));
    context.reset(engine->createExecutionContext());
}

size_t getSizeByDim(const nvinfer1::Dims & dims)
{
    size_t size =1;
    for (size_t i= 0; i < dims.nbDims; i++) {
        size*=dims.d[i];
    }
    return size;
}
int main(){
    std::string model_path = "/home/atharva/fasterrcnn.onnx";
    std::string image = "/home/atharva/img.jpeg";
    int batch = 1;
    TRTUniquePtr<nvinfer1::ICudaEngine> engine{nullptr};
    TRTUniquePtr<nvinfer1::IExecutionContext> context{nullptr};
    parseONNX(model_path, engine, context);
    std::cout<<"Parsed Model";

And the output is as follows -


----------------------------------------------------------------
Input filename:   /home/atharva/fasterrcnn.onnx
ONNX IR version:  0.0.6
Opset version:    11
Producer name:    pytorch
Producer version: 1.6
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
ERROR: builtin_op_importers.cpp:2179 In function importPad:
[8] Assertion failed: inputs.at(1).is_weights()
Error could not parseParsed Model
Process finished with exit code 0

Hi @atharva362,

I am afraid, but TRT currently does not support convolutions where the weights are tensors.

Thanks!

Thanks for replying @AakankshaS
I will probably go ahead with loading a pretrained caffe model now…

Hello @AakankshaS, I tried using the SSD model(As provided in the data folder in tensorRT examples) and use the following lines to parse the model(in place of Onnx parser)


    auto parser = TRTUniquePtr<nvcaffeparser1::ICaffeParser>(nvcaffeparser1::createCaffeParser());
    if (!parser)
    {
        std::cout<<"Error parsing";
        exit(-1);
    }
    auto model = parser->parse("/home/atharva/deploy.prototxt", "/home/atharva/ssd.caffemodel", *network, nvinfer1::DataType::kFLOAT);

It displays the following message - could not parse layer type Normalize and it ends with code 139(signal 11 SIGSEV)… I first thought this was due to error in loading the image, so I commented the lines out the error still persists…
How could I work around this?

Hi @atharva362,
Request you to check the below link

Thanks!

Hi, I am facing the same problem, I have a Pytorch model and convert it to ONNX then I run deepstream to build TensorRT engine file but it raise above error. How can I fix it ? I have to convert Pytorch to Caffe then from Caffe to TensorRT ?

1 Like