Hello,
I modify the TensorRT python sample in samples/python/fc_plugin_caffe_mnist to support serializing and deserializing. But when I deserialize the saved engine file, it crashes and produces too much log.
load_engine
Reading engine from file mnist.engine
<class 'build.fcplugin.FCpluginFactory'>
Traceback (most recent call last):
File "sample2.py", line 156, in <module>
des_engine()
File "sample2.py", line 151, in des_engine
engine = runtime.deserialize_cuda_engine(f.read(), fc_factory)
TypeError: deserialize_cuda_engine(): incompatible funtion arguments. The following argument types are supported:
1. (self: tensorrt.tensorrt.Runtime, serialized_engine: buffer, plugin_factory: tensorrt.tensorrt.IPluginFactory = None) -> tensorrt.tensorrt.ICudaEngine
Invoked with: <tensorrt.tensorrt.Runtime object at 0x7f2b50478298, '\xd8l\x1a\x00\x00\x00 ...............'>
This gist https://gist.github.com/crouchggj/63ebd84193ff4a695efe82c9b1d54f82 or attachment file is my modified test source code which you can test. I use main() function to save engined file
def main():
# Get data files for the model.
data_path, [deploy_file, model_file, mean_proto] = common.find_sample_data(description="Runs an MNIST network using a Caffe model file", subfolder="mnist", find_files=["mnist.prototxt", "mnist.caffemodel", "mnist_mean.binaryproto"])
with build_engine(deploy_file, model_file) as engine:
# Build an engine, allocate buffers and create a stream.
# For more information on buffer allocation, refer to the introductory samples.
print("save engine")
buf = engine.serialize()
print (type(buf))
with open("mnist.engine", 'wb') as f:
f.write(buf)
f.close()
and des_engine() function to deserialize engine file.
def des_engine():
print("load enine")
engine_file_path = "mnist.engine"
if os.path.exists(engine_file_path):
print("Reading engine from file {}".format(engine_file_path))
with open(engine_file_path, "rb") as f, trt.Runtime(TRT_LOGGER) as runtime:
print (type(fc_factory))
engine = runtime.deserialize_cuda_engine(f.read(), fc_factory)
print("success!")
I test it in TensorRT5 RC and TensorRT5 GA, they produce same error.
My environment:
CUDA version: CUDA10
CUDNN version: 7.3.1
Python version: 2.7
TensorRT version: TensorRT 5 RC or TensorRT 5 GA
sample2.py.zip (3.46 KB)