Description
Hi,
I am working on a project in which I trained a FCN8-ResNet18 model (thanks to this repository) using PyTorch.
Now I want to run my model on a Jetson Nano and I would like to optimize performance as mentionned in @dusty_nv’s article (here) thus I want to convert my ONNX model to TensorRT.
I used this repository to convert my model.
But it didn’t work.
The code used to convert my PyTorch model to ONNX locally :
# Export the model
torch.onnx.export(model_loaded, # model being run
x, # model input (or a tuple for multiple inputs)
"fcn8_resnet18.onnx", # where to save the model (can be a file or file-like object)
export_params=True, # store the trained parameter weights inside the model file
opset_version=10, # the ONNX version to export the model to
do_constant_folding=True, # whether to execute constant folding for optimization
input_names = ['input'], # the model's input names
output_names = ['output'], # the model's output names
dynamic_axes={'input' : {0 : 'batch_size'}, # variable lenght axes
'output' : {0 : 'batch_size'}})
The command used to convert my ONNX model to TensorRT on the Jetson Nano :
onnx2trt fcn8_resnet18.onnx -o fc8_resnet18.trt
The output :
----------------------------------------------------------------
Input filename: fcn8_resnet18.onnx
ONNX IR version: 0.0.6
Opset version: 10
Producer name: pytorch
Producer version: 1.6
Domain:
Model version: 0
Doc string:
----------------------------------------------------------------
Parsing model
[2020-10-19 09:22:51 WARNING] [TRT]/home/lixo/background_substractor/onnx-tensorrt/onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
Building TensorRT engine, FP16 available:1
Max batch size: 32
Max workspace size: 1024 MiB
[2020-10-19 09:22:51 ERROR] Network has dynamic or shape inputs, but no optimization profile has been defined.
[2020-10-19 09:22:51 ERROR] Network validation failed.
terminate called after throwing an instance of 'std::runtime_error'
what(): Failed to create object
Aborted (core dumped)
I have also tried this python script on Jetson Nano :
def build_engine(onnx_file_path):
# initialize TensorRT engine and parse ONNX model
builder = trt.Builder(TRT_LOGGER)
network = builder.create_network()
parser = trt.OnnxParser(network, TRT_LOGGER)
# parse ONNX
with open(onnx_file_path, 'rb') as model:
print('Beginning ONNX file parsing')
parser.parse(model.read())
print('Completed parsing of ONNX file')
# allow TensorRT to use up to 1GB of GPU memory for tactic selection
builder.max_workspace_size = 1 << 30
# we have only one image in batch
builder.max_batch_size = 34
# use FP16 mode if possible
if builder.platform_has_fast_fp16:
builder.fp16_mode = True
# generate TensorRT engine optimized for the target platform
print('Building an engine...')
last_layer = network.get_layer(network.num_layers - 1)
if not last_layer:
network.mark_output(last_layer.get_output(0))
engine = builder.build_cuda_engine(network)
#context = engine.create_execution_context()
print("Completed creating Engine")
return engine
def main():
# initialize TensorRT engine and parse ONNX model
ONNX_FILE_PATH = "fcn8_resnet18.onnx"
engine = build_engine(ONNX_FILE_PATH)
engine_file_path = 'fcn8_resnet18.trt'
with open(engine_file_path, "wb") as f:
f.write(engine.serialize())
if __name__ == "__main__":
# execute only if run as a script
main()
And I got the following output :
Beginning ONNX file parsing
Completed parsing of ONNX file
Building an engine...
python3: ../builder/Network.cpp:1187: virtual nvinfer1::ILayer* nvinfer1::Network::getLayer(int) const: Assertion `layerIndex >= 0' failed.
Aborted (core dumped)
Thanks for you help.
Environment
On a Jetson Nano
TensorRT Version: 7.1.3.0
Torchvision Version: 0.7.0a0+78ed10c
Python Version (if applicable): 3.6.9
PyTorch Version (if applicable): 1.6.0
Relevant Files
my ONNX model : [here]