There are memory leaks in the TensorRT runtime, engine and context. Running the full sampleOnnxMNIST using valgrind,
aaron@kinabalu:~/TensorRT-4.0.1.6/targets/x86_64-linux-gnu/samples$ valgrind ./../bin/sample_onnx_mnist
==17944== Memcheck, a memory error detector
==17944== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==17944== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==17944== Command: ./../bin/sample_onnx_mnist
==17944==
==17944== Warning: set address range perms: large range [0x14475000, 0x25910000) (defined)
==17944== Warning: noted but unhandled ioctl 0x30000001 with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==17944== Warning: noted but unhandled ioctl 0x27 with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==17944== Warning: noted but unhandled ioctl 0x7ff with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==17944== Warning: noted but unhandled ioctl 0x25 with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==17944== Warning: noted but unhandled ioctl 0x37 with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==17944== Warning: noted but unhandled ioctl 0x17 with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==17944== Warning: set address range perms: large range [0x200000000, 0x300200000) (noaccess)
==17944== Warning: set address range perms: large range [0x10006000000, 0x10106000000) (noaccess)
==17944== Warning: noted but unhandled ioctl 0x19 with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==17944== Warning: set address range perms: large range [0x10106000000, 0x10206000000) (noaccess)
==17944== Warning: noted but unhandled ioctl 0x21 with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==17944== Warning: noted but unhandled ioctl 0x1b with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==17944== Warning: noted but unhandled ioctl 0x42 with no size/direction hints.
==17944== This could cause spurious value errors to appear.
==17944== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
---------------------------
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@+ :@@@@@@@@
@@@@@@@@@@@@@@%= :. --%@@@@@
@@@@@@@@@@@@@%. -@= - :@@@@@
@@@@@@@@@@@@@: -@@#%@@ #@@@@
@@@@@@@@@@@@: #@@@@@@@-#@@@@
@@@@@@@@@@@= #@@@@@@@@=%@@@@
@@@@@@@@@@= #@@@@@@@@@:@@@@@
@@@@@@@@@+ -@@@@@@@@@%.@@@@@
@@@@@@@@@::@@@@@@@@@@+-@@@@@
@@@@@@@@-.%@@@@@@@@@@.*@@@@@
@@@@@@@@ *@@@@@@@@@@@ *@@@@@
@@@@@@@% %@@@@@@@@@%.-@@@@@@
@@@@@@@:*@@@@@@@@@+. %@@@@@@
@@@@@@# @@@@@@@@@# .*@@@@@@@
@@@@@@# @@@@@@@@= +@@@@@@@@
@@@@@@# @@@@@@%. .+@@@@@@@@@
@@@@@@# @@@@@*. -%@@@@@@@@@@
@@@@@@# --- =@@@@@@@@@@@@
@@@@@@# *%@@@@@@@@@@@@@
@@@@@@@%: -=%@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Prob 0 0.99980: **********
Prob 1 0.00001:
Prob 2 0.00002:
Prob 3 0.00003:
Prob 4 0.00004:
Prob 5 0.00005:
Prob 6 0.00026:
Prob 7 0.00007:
Prob 8 0.00008:
Prob 9 0.00009:
==17944==
==17944== HEAP SUMMARY:
==17944== in use at exit: 783,493,472 bytes in 471,737 blocks
==17944== total heap usage: 854,116 allocs, 382,379 frees, 1,070,313,665 bytes allocated
==17944==
==17944== LEAK SUMMARY:
==17944== definitely lost: 1,664 bytes in 18 blocks
==17944== indirectly lost: 55 bytes in 1 blocks
==17944== possibly lost: 2,180,612 bytes in 16,255 blocks
==17944== still reachable: 781,311,141 bytes in 455,463 blocks
==17944== of which reachable via heuristic:
==17944== stdstring : 6,373 bytes in 119 blocks
==17944== suppressed: 0 bytes in 0 blocks
==17944== Rerun with --leak-check=full to see details of leaked memory
==17944==
==17944== For counts of detected and suppressed errors, rerun with: -v
==17944== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
There are still 471,737 blocks at exit. Running the inference more than 1 time does not increase leak.
I have isolated it down to a minimal example, most of the leaks are coming from the onnxToTRTModel, runtime, engine and context.
int main(int argc, char** argv)
{
// create a TensorRT model from the onnx model and serialize it to a stream
IHostMemory *trtModelStream{nullptr};
onnxToTRTModel("mnist.onnx", 1, trtModelStream); //471726
assert(trtModelStream != nullptr);
// deserialize the engine
IRuntime* runtime = createInferRuntime(gLogger);
assert(runtime != nullptr);
ICudaEngine* engine = runtime->deserializeCudaEngine(trtModelStream->data(), trtModelStream->size(), nullptr); //471730
assert(engine != nullptr);
IExecutionContext *context = engine->createExecutionContext(); //471734
assert(context != nullptr);
// destroy the engine
trtModelStream->destroy();
context->destroy();
engine->destroy();
runtime->destroy();
return 0;
}
aaron@kinabalu:~/TensorRT-4.0.1.6/targets/x86_64-linux-gnu/samples$ valgrind ./../bin/sample_mem
==18766== Memcheck, a memory error detector
==18766== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==18766== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==18766== Command: ./../bin/sample_mem
==18766==
==18766== Warning: set address range perms: large range [0x13b43000, 0x24fde000) (defined)
==18766== Warning: noted but unhandled ioctl 0x30000001 with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766== Warning: noted but unhandled ioctl 0x27 with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766== Warning: noted but unhandled ioctl 0x7ff with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766== Warning: noted but unhandled ioctl 0x25 with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766== Warning: noted but unhandled ioctl 0x37 with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766== Warning: noted but unhandled ioctl 0x17 with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766== Warning: set address range perms: large range [0x200000000, 0x300200000) (noaccess)
==18766== Warning: set address range perms: large range [0x10006000000, 0x10106000000) (noaccess)
==18766== Warning: noted but unhandled ioctl 0x19 with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766== Warning: set address range perms: large range [0x10106000000, 0x10206000000) (noaccess)
==18766== Warning: noted but unhandled ioctl 0x21 with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766== Warning: noted but unhandled ioctl 0x1b with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766== Warning: noted but unhandled ioctl 0x42 with no size/direction hints.
==18766== This could cause spurious value errors to appear.
==18766== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==18766==
==18766== HEAP SUMMARY:
==18766== in use at exit: 783,492,081 bytes in 471,726 blocks
==18766== total heap usage: 842,897 allocs, 371,171 frees, 1,069,481,780 bytes allocated
==18766==
==18766== LEAK SUMMARY:
==18766== definitely lost: 896 bytes in 10 blocks
==18766== indirectly lost: 55 bytes in 1 blocks
==18766== possibly lost: 2,180,692 bytes in 16,256 blocks
==18766== still reachable: 781,310,438 bytes in 455,459 blocks
==18766== of which reachable via heuristic:
==18766== stdstring : 6,373 bytes in 119 blocks
==18766== suppressed: 0 bytes in 0 blocks
==18766== Rerun with --leak-check=full to see details of leaked memory
==18766==
==18766== For counts of detected and suppressed errors, rerun with: -v
==18766== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
The compilation have been done using the sample makefiles with minor edits. Does anyone have a solution for this?