Details of Setup
- Hardware Platform (Jetson / GPU): GPU
- DeepStream Version: 7.1.0
- TensorRT Version: 10.7.0.23-1+cuda12.6 amd64
- NVIDIA GPU Driver Version (valid for GPU only): 560.35.03 (CUDA Version: 12.6)
Issue Type: Bug
Description of Issue:
When using either resnet50_market1501.etlt or the open source ghost_reid.onnx Re-ID Models for tracking I’m getting the following backtraces:
resnet:
[New Thread 0x756de9e00640 (LWP 286)]
[New Thread 0x756deea00640 (LWP 287)]
[New Thread 0x756def400640 (LWP 288)]
0:01:00.835279545 141 0x62631d2f37c0 INFO nvinfer gstnvinfer_impl.cpp:343:notifyLoadModelStatus:<lp_detector> [UID 2]: Load new model:/home/ds_tracker/component_configs/config_infer_secondary_yoloV7_lp.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
~~ CLOG[src/modules/ReID/ReID.cpp, loadTRTEngine() @line 552]: Engine file does not exist
[NvMultiObjectTracker] Load engine failed. Create engine again.
WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:922 INT8 calibration file not specified. Trying FP16 mode.
[NvMultiObjectTracker] Serialized plan file cached at location: /home/ds_tracker/models/ReID/resnet50_market1501.etlt_b100_gpu0_fp16.engine
[NvMultiObjectTracker] Initialized
[New Thread 0x756defe00640 (LWP 329)]
[New Thread 0x756e01800640 (LWP 330)]
Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x0000756ed081e550 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) backtrace
#0 0x0000756ed081e550 in () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1 0x0000756ed081e8b1 in () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#2 0x0000756ed081e9c7 in __gxx_personality_v0 () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x0000756eceb96fe9 in __libunwind_Unwind_Resume () at /usr/lib/x86_64-linux-gnu/libunwind.so.8
#4 0x0000756e5b7e1d1e in () at /usr/lib/x86_64-linux-gnu/libnvinfer.so.10
#5 0x0000756e5c6ebacc in () at /usr/lib/x86_64-linux-gnu/libnvinfer.so.10
#6 0x0000756e5c6ec260 in () at /usr/lib/x86_64-linux-gnu/libnvinfer.so.10
#7 0x0000756e5c7269c8 in () at /usr/lib/x86_64-linux-gnu/libnvinfer.so.10
#8 0x0000756e5c727426 in () at /usr/lib/x86_64-linux-gnu/libnvinfer.so.10
#9 0x0000756e5c7274be in () at /usr/lib/x86_64-linux-gnu/libnvinfer.so.10
#10 0x0000756eac79c258 in () at /opt/nvidia/deepstream/deepstream/lib/libnvds_infer.so
#11 0x0000756eac773a56 in nvdsinfer::NvDsInferContextImpl::deserializeEngineAndBackend(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::shared_ptr<nvdsinfer::TrtEngine>&, std::unique_ptr<nvdsinfer::BackendContext, std::default_delete<nvdsinfer::BackendContext> >&) () at /opt/nvidia/deepstream/deepstream/lib/libnvds_infer.so
#12 0x0000756eac774873 in nvdsinfer::NvDsInferContextImpl::generateBackendContext(_NvDsInferContextInitParams&) ()
at /opt/nvidia/deepstream/deepstream/lib/libnvds_infer.so
#13 0x0000756eac77b774 in nvdsinfer::NvDsInferContextImpl::initialize(_NvDsInferContextInitParams&, void*, void (*)(INvDsInferContext*, unsigned int, NvDsInferLogLevel, char const*, void*)) () at /opt/nvidia/deepstream/deepstream/lib/libnvds_infer.so
#14 0x0000756eac77c0ee in createNvDsInferContext(INvDsInferContext**, _NvDsInferContextInitParams&, void*, void (*)(INvDsInferContext*, unsigned int, NvDsInferLogLevel, char const*, void*)) () at /opt/nvidia/deepstream/deepstream/lib/libnvds_infer.so
#15 0x0000756eaf0cff48 in () at /opt/nvidia/deepstream/deepstream/lib/gst-plugins/libnvdsgst_infer.so
#16 0x0000756ecea07441 in () at /usr/lib/x86_64-linux-gnu/libgstbase-1.0.so.0
#17 0x0000756ecea07675 in () at /usr/lib/x86_64-linux-gnu/libgstbase-1.0.so.0
#18 0x0000756ecec677df in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#19 0x0000756ecec6808c in gst_pad_set_active () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#20 0x0000756ecec4f1f5 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#21 0x0000756ecec5840b in gst_iterator_fold () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#22 0x0000756ececc634a in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#23 0x0000756ecec4caf6 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#24 0x0000756ecec4f4b0 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#25 0x0000756ecec4e7c9 in gst_element_change_state () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#26 0x0000756ecec4eed5 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#27 0x0000756ecec2788c in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#28 0x0000756ecec7a826 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#29 0x0000756ecec4e7c9 in gst_element_change_state () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#30 0x0000756ecec4e80f in gst_element_change_state () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#31 0x0000756ecec4eed5 in () at /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0
#32 0x0000756ed135fe2e in () at /usr/lib/x86_64-linux-gnu/libffi.so.8
#33 0x0000756ed135c493 in () at /usr/lib/x86_64-linux-gnu/libffi.so.8
#34 0x0000756ed0ec5722 in () at /usr/lib/python3/dist-packages/gi/_gi.cpython-310-x86_64-linux-gnu.so
#35 0x0000756ed0ec3826 in () at /usr/lib/python3/dist-packages/gi/_gi.cpython-310-x86_64-linux-gnu.so
#36 0x0000756ed0ec3a7d in () at /usr/lib/python3/dist-packages/gi/_gi.cpython-310-x86_64-linux-gnu.so
#37 0x00006262e5e2639b in _PyObject_MakeTpCall ()
#38 0x00006262e5e20009 in _PyEval_EvalFrameDefault ()
#39 0x00006262e5e3038c in _PyFunction_Vectorcall ()
#40 0x00006262e5e1a97f in _PyEval_EvalFrameDefault ()
#41 0x00006262e5e3038c in _PyFunction_Vectorcall ()
#42 0x00006262e5e1a790 in _PyEval_EvalFrameDefault ()
#43 0x00006262e5effa16 in ()
#44 0x00006262e5eff8e6 in PyEval_EvalCode ()
#45 0x00006262e5f26108 in ()
#46 0x00006262e5f2090f in ()
#47 0x00006262e5f25ea5 in ()
#48 0x00006262e5f25488 in _PyRun_SimpleFileObject ()
#49 0x00006262e5f25167 in _PyRun_AnyFileObject ()
#50 0x00006262e5f198ae in Py_RunMain ()
#51 0x00006262e5ef393d in Py_BytesMain ()
#52 0x0000756ed165cd90 in () at /usr/lib/x86_64-linux-gnu/libc.so.6
#53 0x0000756ed165ce40 in __libc_start_main () at /usr/lib/x86_64-linux-gnu/libc.so.6
#54 0x00006262e5ef3835 in _start ()
onnx:
0:00:53.335015207 2598 0x60082607f3e0 INFO nvinfer gstnvinfer_impl.cpp:343:notifyLoadModelStatus:<lp_detector> [UID 2]: Load new model:/home/ds_tracker/component_configs/config_infer_secondary_yoloV7_lp.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
[NvMultiObjectTracker] Initialized
[New Thread 0x7e0a01400640 (LWP 2718)]
[New Thread 0x7e0a20e00640 (LWP 2719)]
Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00007e0acbc99550 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
What may cause this issue?