tensorrt - 4.0.1
Cuda - 9.0
cudnn - 7.1.3
I am receiving the following segmentation fault with particular input size of tensorrt engine.
The engine gets generated and loaded successfully. Generation of 3 types of engine is done. Inference on 2 engine works. On same pipeline, there a error for highest input size engine.
(gdb) bt
#0 0x00007fffcf5589d2 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#1 0x00007fffcf606abe in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#2 0x00007fffcf6db4c0 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#3 0x00007fffcf608400 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#4 0x00007fffcf51ed34 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#5 0x00007fffcf522629 in ?? () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#6 0x00007fffcf66e195 in cuMemcpyDtoH_v2 () from /usr/lib/x86_64-linux-gnu/libcuda.so.1
#7 0x00007fffe34b860a in ?? () from /usr/local/cuda-9.0/lib64/libcudart.so.9.0
#8 0x00007fffe3498d76 in ?? () from /usr/local/cuda-9.0/lib64/libcudart.so.9.0
#9 0x00007fffe34be43b in cudaMemcpy () from /usr/local/cuda-9.0/lib64/libcudart.so.9.0
#10 0x00000000004571a5 in basic_cuda_tensor<char, (unsigned char)1>::partialToHost (this=0x7fffa86f9970, buffer=0x7fffd2bb2010, batch_size=1, max_batch_size=1)
at /app/3rdparty/sources/XXX/YYY/xyz1.cpp:67
#11 0x00000000004548d5 in somefunction::operator() (this=0x6e7c840, inputs=std::vector of length 1, capacity 1 = {...}, outputs=std::vector of length 2, capacity 2 = {...}, batch_size=1)
at /app/3rdparty/sources/sources/XXX/YYY/xyz2.cpp:312
#12 0x000000000043a24b in somefunction::somefunction[abi:cxx11](somefunction::somefunction&, somefunction&, somefunction, cv::Mat) (this=0x94aaf0, in=..., handle=..., count=1, file=...)
at /app/3rdparty/sources/XXX/YYY/abc.cpp:117
#13 0x000000000043af44 in somefunction::somefunction[abi:cxx11](cv::Mat, int) (this=0x94aaf0, file=..., i=0) at /app/3rdparty/sources/XXX/YYY/abc.cpp:226
#14 0x000000000040a95c in main (argc=1, argv=0x7fffffffe4a0) at /app/3rdparty/sources/XXX/YYY/abc.cpp:416
Considering all the different resolution works. So there is nothing wrong with the input and configuration. Also on this particular inference part is also called for them.
What is issue and why it happens for a particular engine ?