Hi All,
• Hardware Platform (GPU V100)
• DeepStream Version 5.0
• TensorRT Version 7.0.0.11-1
• NVIDIA GPU Driver Version (valid for GPU only) 440.64.00
• Cuda version 10.2
• Cudnn version 7
I have a VM with V100 by Azure on which I want to run inference from a model I have trained in Transfer Learning Toolkit(TLT). So I have installed deepstream on it following this link. I am trying out deepstream by running one(source30_1080p_dec_infer-resnet_tiled_display_int8.txt) of the config files, using the deepstream-app -c command. The output is below:-
/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app$ deepstream-app -c source30_1080p_dec_infer-resnet_tiled_display_int8.txt (gst-plugin-scanner:83713): GStreamer-WARNING **: 17:19:32.971: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtrtserver.so: cannot open shared object file: No such file or directory libEGL warning: DRI2: failed to authenticate ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:1408 Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine open error 0:00:02.079019906 83712 0x556d7a3424d0 WARN nvinfer gstnvinfer.cpp:599:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1566> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine failed 0:00:02.079052504 83712 0x556d7a3424d0 WARN nvinfer gstnvinfer.cpp:599:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1673> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine failed, try rebuild 0:00:02.079066104 83712 0x556d7a3424d0 INFO nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1591> [UID = 1]: Trying to create engine from model files INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:37 [TRT]: Reading Calibration Cache for calibrator: EntropyCalibration2 INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:37 [TRT]: Generated calibration scales using calibration cache. Make sure that calibration cache has latest scales. INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:37 [TRT]: To regenerate calibration cache, please delete the existing one. TensorRT will generate a new calibration cache. INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:37 [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. INFO: ../nvdsinfer/nvdsinfer_func_utils.cpp:37 [TRT]: Detected 1 inputs and 2 output network tensors. 0:00:06.635185870 83712 0x556d7a3424d0 INFO nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1624> [UID = 1]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-5.0/samples/models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine successfully WARNING: ../nvdsinfer/nvdsinfer_func_utils.cpp:34 [TRT]: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:685 [Implicit Engine Info]: layers num: 3 0 INPUT kFLOAT input_1 3x368x640 1 OUTPUT kFLOAT conv2d_bbox 16x23x40 2 OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40 0:00:06.643602190 83712 0x556d7a3424d0 INFO nvinfer gstnvinfer_impl.cpp:311:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/config_infer_primary.txt sucessfully Runtime commands: h: Print this help q: Quit p: Pause r: Resume NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source. To go back to the tiled display, right-click anywhere on the window. **PERF: FPS 0 (Avg) FPS 1 (Avg) FPS 2 (Avg) FPS 3 (Avg) FPS 4 (Avg) FPS 5 (Avg) FPS 6 (Avg) FPS 7 (Avg) FPS 8 (Avg) FPS 9 (Avg) FPS 10 (Avg) FPS 11 (Avg) FPS 12 (Avg) FPS 13 (Avg) FPS 14 (Avg) FPS 15 (Avg) FPS 16 (Avg) FPS 17 (Avg) FPS 18 (Avg) FPS 19 (Avg) FPS 20 (Avg) FPS 21 (Avg) FPS 22 (Avg) FPS 23 (Avg) FPS 24 (Avg) FPS 25 (Avg) FPS 26 (Avg) FPS 27 (Avg) FPS 28 (Avg) FPS 29 (Avg) **PERF: 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) 0.00 (0.00) ** INFO: <bus_callback:181>: Pipeline ready cuGraphicsGLRegisterBuffer failed with error(219) gst_eglglessink_cuda_init texture = 1 0:00:07.663188381 83712 0x556d6cb2b1e0 WARN nvinfer gstnvinfer.cpp:1946:gst_nvinfer_output_loop:<primary_gie> error: Internal data stream error. 0:00:07.663209181 83712 0x556d6cb2b1e0 WARN nvinfer gstnvinfer.cpp:1946:gst_nvinfer_output_loop:<primary_gie> error: streaming stopped, reason not-negotiated (-4) ERROR from primary_gie: Internal data stream error. Debug info: gstnvinfer.cpp(1946): gst_nvinfer_output_loop (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie: streaming stopped, reason not-negotiated (-4) Quitting ERROR from sink_bin_queue: Internal data stream error. Debug info: gstqueue.c(988): gst_queue_handle_sink_event (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstQueue:sink_bin_queue: streaming stopped, reason not-negotiated (-4) ERROR from qtdemux25: Internal data stream error. Debug info: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin25/GstURIDecodeBin:src_elem/GstDecodeBin:decodebin25/GstQTDemux:qtdemux25: streaming stopped, reason not-negotiated (-4) ERROR from qtdemux4: Internal data stream error. Debug info: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin5/GstURIDecodeBin:src_elem/GstDecodeBin:decodebin5/GstQTDemux:qtdemux4: streaming stopped, reason not-negotiated (-4) ERROR from qtdemux2: Internal data stream error. Debug info: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin7/GstURIDecodeBin:src_elem/GstDecodeBin:decodebin7/GstQTDemux:qtdemux2: streaming stopped, reason not-negotiated (-4) ERROR from qtdemux3: Internal data stream error. Debug info: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin1/GstURIDecodeBin:src_elem/GstDecodeBin:decodebin1/GstQTDemux:qtdemux3: streaming stopped, reason not-negotiated (-4) App run failed
On running the above command with sudo am getting the following errors:-
/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app$ sudo deepstream-app -c source30_1080p_dec_infer-resnet_tiled_display_int8.txt libEGL warning: DRI2: failed to authenticate WARNING: ../nvdsinfer/nvdsinfer_func_utils.cpp:34 [TRT]: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles 0:00:02.027328366 85475 0x5631edb63ad0 INFO nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1577> [UID = 1]: deserialized trt engine from :/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:685 [Implicit Engine Info]: layers num: 3 0 INPUT kFLOAT input_1 3x368x640 1 OUTPUT kFLOAT conv2d_bbox 16x23x40 2 OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40 0:00:02.027426963 85475 0x5631edb63ad0 INFO nvinfer gstnvinfer.cpp:602:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1681> [UID = 1]: Use deserialized engine model: /opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine 0:00:02.030956949 85475 0x5631edb63ad0 INFO nvinfer gstnvinfer_impl.cpp:311:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/config_infer_primary.txt sucessfully Runtime commands: h: Print this help q: Quit p: Pause r: Resume NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source. To go back to the tiled display, right-click anywhere on the window. ** INFO: <bus_callback:181>: Pipeline ready cuGraphicsGLRegisterBuffer failed with error(219) gst_eglglessink_cuda_init texture = 1 0:00:02.853179252 85475 0x5631e069fde0 WARN nvinfer gstnvinfer.cpp:1946:gst_nvinfer_output_loop:<primary_gie> error: Internal data stream error. 0:00:02.853203051 85475 0x5631e069fde0 WARN nvinfer gstnvinfer.cpp:1946:gst_nvinfer_output_loop:<primary_gie> error: streaming stopped, reason not-negotiated (-4) ERROR from primary_gie: Internal data stream error. Debug info: gstnvinfer.cpp(1946): gst_nvinfer_output_loop (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie: streaming stopped, reason not-negotiated (-4) Quitting ERROR from sink_bin_queue: Internal data stream error. Debug info: gstqueue.c(988): gst_queue_handle_sink_event (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstQueue:sink_bin_queue: streaming stopped, reason not-negotiated (-4) ERROR from qtdemux3: Internal data stream error. Debug info: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem/GstDecodeBin:decodebin0/GstQTDemux:qtdemux3: streaming stopped, reason not-negotiated (-4) App run failed
I know deepstream-app command is working because am getting the following output to deepstream-app --help :-
deepstream-app --help Usage: deepstream-app [OPTION?] Nvidia DeepStream Demo Help Options: -h, --help Show help options --help-all Show all help options --help-gst Show GStreamer Options Application Options: -v, --version Print DeepStreamSDK version -t, --tiledtext Display Bounding box labels in tiled mode --version-all Print DeepStreamSDK and dependencies version -c, --cfg-file Set the config file -i, --input-file Set the input file
Is there anything I am missing? Any help would be very appreciated. Thanks.
Regards.