Memory error when I return the trt engine - runtime.deserialize_cuda_engine(f.read())

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

What I’m using:

  • nvidia-docker 19.03-py3
  • mxnet-cu101==1.4.1

I exported a model from MxNet to ONNX, and now I’m trying to import to TRT.
The error below appears when I’m trying to return the serialized engine that was build from the provided .onnx file.

Any files or reproduceable code that is needed, I can provide here or in private (preferred in private).

When using the code:

if os.path.exists(engine_file_path):
        # If a serialized engine exists, use it instead of building an engine.
        print("Reading engine from file {}".format(engine_file_path))
        with open(engine_file_path, "rb") as f, trt.Runtime(TRT_LOGGER) as runtime:
             return runtime.deserialize_cuda_engine(f.read())

Would love to know why this error happens:

terminate called after throwing an instance of 'dmlc::Error'
  what():  [08:22:27] src/storage/./pooled_storage_manager.h:92: CUDA: invalid argument

Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x40ba6a) [0x7f90c7cb1a6a]
[bt] (1) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x40c081) [0x7f90c7cb2081]
[bt] (2) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x3423ce1) [0x7f90cacc9ce1]
[bt] (3) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x34280ec) [0x7f90cacce0ec]
[bt] (4) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x34286c7) [0x7f90cacce6c7]
[bt] (5) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x40f41a) [0x7f90c7cb541a]
[bt] (6) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x34274ec) [0x7f90caccd4ec]
[bt] (7) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x40f41a) [0x7f90c7cb541a]
[bt] (8) /lib/x86_64-linux-gnu/libc.so.6(+0x39ff8) [0x7f918674cff8]
[bt] (9) /lib/x86_64-linux-gnu/libc.so.6(+0x3a045) [0x7f918674d045]

Segmentation fault: 11

Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x40ba6a) [0x7f90c7cb1a6a]
[bt] (1) /usr/local/lib/python3.5/dist-packages/mxnet/libmxnet.so(+0x342cd06) [0x7f90cacd2d06]
[bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x354b0) [0x7f91867484b0]
[bt] (3) /lib/x86_64-linux-gnu/libc.so.6(abort+0x2d6) [0x7f918674a196]
[bt] (4) /usr/lib/x86_64-linux-gnu/libstdc++.so.6(__gnu_cxx::__verbose_terminate_handler()+0x16d) [0x7f914f7d484d]
[bt] (5) /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d6b6) [0x7f914f7d26b6]
[bt] (6) /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8c6a9) [0x7f914f7d16a9]
[bt] (7) /usr/lib/x86_64-linux-gnu/libstdc++.so.6(__gxx_personality_v0+0x2e5) [0x7f914f7d2005]
[bt] (8) /lib/x86_64-linux-gnu/libgcc_s.so.1(+0xff83) [0x7f914f53ef83]
[bt] (9) /lib/x86_64-linux-gnu/libgcc_s.so.1(_Unwind_Resume+0x57) [0x7f914f53f487]