I am trying to convert LOFTR pytorch model to tensorrt. The model is just combination of CNN with Transformers, but the implementation uses a lot of einsum operation so this is what I think gives me several errors. Because I tested the entire pipeline on other models and it worked well, so I don’t think I have errors in the code.
This is the process.
I converted model to the onnx and tested it using onnxruntime, it works well. Initially I used
opset = 15, but now decreased to 12. Results are same for both versions.
with torch.no_grad(): torch.onnx.export(model, input_dict, out_filename, do_constant_folding=True, verbose=True, opset_version=12)
This is the code for onnx conversion.
Then I use the generated file for trt conversion. During parsing the onnx file I get the following error,
[TRT] [E] [layers.cpp::EinsumLayer::5821] Error Code 3: API Usage Error (Parameter check failed at: optimizer/api/layers.cpp::EinsumLayer::5821, condition: nbInputs > 0 && nbInputs <= MAX_EINSUM_NB_INPUTS )
but it finishes the parsing and continues to build the model.
plan = builder.build_serialized_network(network, config) with trt.Runtime(TRT_LOGGER) as runtime: engine = runtime.deserialize_cuda_engine(plan)
I use this script, for generation but it produced error of
network.cpp::validate::2726] Error Code 4: Internal Error (Network must have at least one output
After this I was curious to look if the model is parsed correctly at all and when I print
after model parsing, it returns 0. I also added following line to my model
network.mark_output(network.get_layer(network.num_layers - 1).get_output(0))
This produces the following error
[TRT] [E] 2: [checkSanity.cpp::checkSanity::94] Error Code 2: Internal Error (Assertion tensors.size() == g.tensors.size() failed.
I got stuck here, I am not sure which one I have to fix, whether the error is in parsing or in generating step.
TensorRT Version: 18.104.22.168
GPU Type: RTX 2080
Nvidia Driver Version: 460.73.01
CUDA Version: 11.2
Operating System + Version: “Debian GNU/Linux 10”
Python Version (if applicable): 3.9
PyTorch Version (if applicable): 1.11.0