I am trying to build a refittable engine from the resnet50 uff model provided in the TensorRT samples. However, when I set
builder.refittable = True. The model fails to build and I get the following error:
root@44e0dfd5ebc5:/workspace/tensorrt/samples/python/introductory_parser_samples# python3 uff_resnet50.py
python3: …/builder/cudnnBuilderGraphNodes.h:217: void nvinfer1::builder::fromSymbolicHelper(nvinfer1::builder::Graph&, Node_&) [with Node_ = nvinfer1::builder::RefittableNodeT<(nvinfer1::builder::NodeType)3, nvinfer1::PoolingParameters, true, 479>]: Assertion `node.expr != nullptr && “node does not have an expression to flip from”’ failed.
Aborted (core dumped)
Code for resnet50 UFF model:
def build_engine_uff(model_file): # You can set the logger severity higher to suppress messages (or lower to display more messages). with trt.Builder(TRT_LOGGER) as builder, builder.create_network() as network, trt.UffParser() as parser,builder.create_builder_config() as builder_config: # Workspace size is the maximum amount of memory available to the builder while building an engine. # It should generally be set as high as possible. builder.refittable = True builder.max_workspace_size = common.GiB(1) # We need to manually register the input and output nodes for UFF. parser.register_input(ModelData.INPUT_NAME, ModelData.INPUT_SHAPE) parser.register_output(ModelData.OUTPUT_NAME) # Load the UFF model and parse it in order to populate the TensorRT network. parser.parse(model_file, network) # Build and return an engine. return builder.build_cuda_engine(network)
However, when I tried the same with the resnet50 onnx example the engine builds with no errors.
Code for ONNX model
def build_engine_onnx(model_file): with trt.Builder(TRT_LOGGER) as builder,builder.create_builder_config() as builder_config, builder.create_network(common.EXPLICIT_BATCH) as network, trt.OnnxParser(network, TRT_LOGGER) as parser: builder.max_workspace_size = common.GiB(1) builder.refittable=True # Load the Onnx model and parse it in order to populate the TensorRT network. print(os.path.exists(model_file)) with open(model_file, 'rb') as model: x = model.read() if not parser.parse(x): print ('ERROR: Failed to parse the ONNX file.') for error in range(parser.num_errors): print (parser.get_error(error)) return None return builder.build_cuda_engine(network)