Deepstream python app yolov7 integration issue

anmolprakashnitc · March 14, 2023, 11:57am

Deepstream 6.2
Tensorrt 8.5.2.2

YOLOV7 Pytorch: GitHub - WongKinYiu/yolov7: Implementation of paper - YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors

Creating Pipeline 
 
Creating Source 
 
Creating H264Parser 

Creating Decoder 

Creating H264 Encoder
Creating H264 rtppay
Playing file /opt/nvidia/deepstream/deepstream-6.2/samples/streams/sample_720p.h264 
Unknown or legacy key specified 'is-classifier' for group [property]
Unknown or legacy key specified 'disable-output-host-copy' for group [property]
Adding elements to Pipeline 

Linking elements in the Pipeline 


 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***


Starting pipeline 

0:00:00.124878528   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<encoder:sink> Unable to try format: Unknown error -1
0:00:00.124894435   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<encoder:sink> Could not probe minimum capture size for pixelformat YM12
0:00:00.124900222   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<encoder:sink> Unable to try format: Unknown error -1
0:00:00.124905786   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<encoder:sink> Could not probe maximum capture size for pixelformat YM12
0:00:00.124912443   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2395:gst_v4l2_object_add_interlace_mode:0x2e3adb0 Failed to determine interlace mode
0:00:00.124923661   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<encoder:sink> Unable to try format: Unknown error -1
0:00:00.124948528   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<encoder:sink> Could not probe minimum capture size for pixelformat NM12
0:00:00.124954506   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<encoder:sink> Unable to try format: Unknown error -1
0:00:00.124960538   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<encoder:sink> Could not probe maximum capture size for pixelformat NM12
0:00:00.124970089   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2395:gst_v4l2_object_add_interlace_mode:0x2e3adb0 Failed to determine interlace mode
0:00:00.125043606   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<encoder:src> Unable to try format: Unknown error -1
0:00:00.125069421   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<encoder:src> Could not probe minimum capture size for pixelformat H264
0:00:00.125075881   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<encoder:src> Unable to try format: Unknown error -1
0:00:00.125082772   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<encoder:src> Could not probe maximum capture size for pixelformat H264
0:00:00.349515819   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349536879   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat MJPG
0:00:00.349542271   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349547552   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat MJPG
0:00:00.349562198   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349568778   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat AV10
0:00:00.349575283   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349598989   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat AV10
0:00:00.349631007   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349637706   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat DVX5
0:00:00.349643456   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349649592   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat DVX5
0:00:00.349695250   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349702136   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat DVX4
0:00:00.349708033   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349714543   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat DVX4
0:00:00.349760962   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349768533   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat MPG4
0:00:00.349774914   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349782093   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat MPG4
0:00:00.349810581   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349818080   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat MPG2
0:00:00.349824704   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349831926   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat MPG2
0:00:00.349845629   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349853600   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat H265
0:00:00.349860470   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349883454   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat H265
0:00:00.349894824   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349902217   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat VP90
0:00:00.349909009   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349926301   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat VP90
0:00:00.349936258   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349943421   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat VP80
0:00:00.349966674   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.349983243   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat VP80
0:00:00.349994325   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.350001726   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe minimum capture size for pixelformat H264
0:00:00.350008223   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:sink> Unable to try format: Unknown error -1
0:00:00.350015329   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:sink> Could not probe maximum capture size for pixelformat H264
0:00:00.350074721   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:src> Unable to try format: Unknown error -1
0:00:00.350083301   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2942:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:src> Could not probe minimum capture size for pixelformat NM12
0:00:00.350089913   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:3057:gst_v4l2_object_get_nearest_size:<nvv4l2-decoder:src> Unable to try format: Unknown error -1
0:00:00.350097077   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2948:gst_v4l2_object_probe_caps_for_format:<nvv4l2-decoder:src> Could not probe maximum capture size for pixelformat NM12
0:00:00.350107365   979      0x2d5f6c0 WARN                    v4l2 gstv4l2object.c:2395:gst_v4l2_object_add_interlace_mode:0x22114b0 Failed to determine interlace mode
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
0:00:03.634480100   979      0x2d5f6c0 INFO                 nvinfer gstnvinfer.cpp:680:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1909> [UID = 1]: deserialized trt engine from :/workspace/deepstream_python_apps/orginal_model/yolov7.onnx_b1_gpu0_fp16.engine
WARNING: [TRT]: The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 2
0   INPUT  kFLOAT images          3x640x640       
1   OUTPUT kFLOAT output          7               

0:00:03.684944836   979      0x2d5f6c0 INFO                 nvinfer gstnvinfer.cpp:680:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2012> [UID = 1]: Use deserialized engine model: /workspace/deepstream_python_apps/orginal_model/yolov7.onnx_b1_gpu0_fp16.engine
0:00:03.687422341   979      0x2d5f6c0 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:yolov7.txt sucessfully
0:00:03.688296410   979      0x2d5f6c0 WARN                 basesrc gstbasesrc.c:3600:gst_base_src_start_complete:<file-source> pad not activated yet
0:00:03.829339679   979      0x2e7a5e0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1082:gst_v4l2_buffer_pool_start:<encoder:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:03.829886332   979      0x2e7a5e0 WARN            v4l2videodec gstv4l2videodec.c:1836:gst_v4l2_video_dec_decide_allocation:<nvv4l2-decoder> Duration invalid, not setting latency
0:00:03.829945573   979      0x2e7a5e0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1082:gst_v4l2_buffer_pool_start:<nvv4l2-decoder:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:03.830647173   979      0x2e7a760 WARN          v4l2bufferpool gstv4l2bufferpool.c:1533:gst_v4l2_buffer_pool_dqbuf:<nvv4l2-decoder:pool:src> Driver should never set v4l2_buffer.field to ANY
ERROR: [TRT]: 1: [runner.cpp::retrieveOutputTensorResult::611] Error Code 1: Cuda Runtime (invalid argument)
ERROR: nvdsinfer_backend.cpp:506 Failed to enqueue trt inference batch
ERROR: nvdsinfer_context_impl.cpp:1650 Infer context enqueue buffer failed, nvinfer error:NVDSINFER_TENSORRT_ERROR
0:00:04.413861903   979      0x2e7a400 WARN                 nvinfer gstnvinfer.cpp:1388:gst_nvinfer_input_queue_loop:<primary-inference> error: Failed to queue input batch for inferencing
Error: gst-stream-error-quark: Failed to queue input batch for inferencing (1): gstnvinfer.cpp(1388): gst_nvinfer_input_queue_loop (): /GstPipeline:pipeline0/GstNvInfer:primary-inference
ERROR: [TRT]: 1: [runner.cpp::retrieveOutputTensorResult::611] Error Code 1: Cuda Runtime (invalid argument)
ERROR: nvdsinfer_backend.cpp:506 Failed to enqueue trt inference batch
ERROR: nvdsinfer_context_impl.cpp:1650 Infer context enqueue buffer failed, nvinfer error:NVDSINFER_TENSORRT_ERROR
0:00:04.446914962   979      0x2e7a400 WARN                 nvinfer gstnvinfer.cpp:1388:gst_nvinfer_input_queue_loop:<primary-inference> error: Failed to queue input batch for inferencing
ERROR: [TRT]: 1: [runner.cpp::retrieveOutputTensorResult::611] Error Code 1: Cuda Runtime (invalid argument)
ERROR: nvdsinfer_backend.cpp:506 Failed to enqueue trt inference batch
ERROR: nvdsinfer_context_impl.cpp:1650 Infer context enqueue buffer failed, nvinfer error:NVDSINFER_TENSORRT_ERROR
0:00:04.476498147   979      0x2e7a400 WARN                 nvinfer gstnvinfer.cpp:1388:gst_nvinfer_input_queue_loop:<primary-inference> error: Failed to queue input batch for inferencing
ERROR: [TRT]: 1: [runner.cpp::retrieveOutputTensorResult::611] Error Code 1: Cuda Runtime (invalid argument)
ERROR: nvdsinfer_backend.cpp:506 Failed to enqueue trt inference batch
ERROR: nvdsinfer_context_impl.cpp:1650 Infer context enqueue buffer failed, nvinfer error:NVDSINFER_TENSORRT_ERROR
0:00:04.505128056   979      0x2e7a400 WARN                 nvinfer gstnvinfer.cpp:1388:gst_nvinfer_input_queue_loop:<primary-inference> error: Failed to queue input batch for inferencing
ERROR: [TRT]: 1: [runner.cpp::retrieveOutputTensorResult::611] Error Code 1: Cuda Runtime (invalid argument)
ERROR: nvdsinfer_backend.cpp:506 Failed to enqueue trt inference batch
ERROR: nvdsinfer_context_impl.cpp:1650 Infer context enqueue buffer failed, nvinfer error:NVDSINFER_TENSORRT_ERROR
0:00:04.533557646   979      0x2e7a400 WARN                 nvinfer gstnvinfer.cpp:1388:gst_nvinfer_input_queue_loop:<primary-inference> error: Failed to queue input batch for inferencing

mchi · March 15, 2023, 1:23am

Hi @anmolprakashnitc
Could you refer to yolo_deepstream/deepstream_yolo at main · NVIDIA-AI-IOT/yolo_deepstream · GitHub for YoloV7 DeepStream deployment?
And GitHub - NVIDIA-AI-IOT/yolo_deepstream: yolo model qat and deploy with deepstream&tensorrt showed the DeepStream YoloV7 performance.

anmolprakashnitc · March 15, 2023, 11:22am

Getting error : nvdsinfer_backend.cpp:506 Failed to enqueue trt inference batch.

What could be reason for this error??

mchi · March 16, 2023, 1:19am

Do you run the project directly? Please elaborate your repo steps.

I tried it on my side with steps below, it works well on T4 dGPU.

start the docker
docker run --gpus all -it --rm -v /tmp/.X11-unix:/tmp/.X11-unix -v /home/$user/:/home/$user/ -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream nvcr.io/nvidia/deepstream:6.2-devel
downlaod the project into docker
apply the change below to change the “config-file=” from yolov4 to yolov7 config - config_infer_primary_yoloV7.txt
download yolov7.onnx from yolo_deepstream/yolov7_qat at main · NVIDIA-AI-IOT/yolo_deepstream · GitHub , and put under yolo_deepstream/deepstream_yolo/
run the application

deepstream-app -c deepstream_app_config_yolo.txt

/// patch

root@36faedb6843e:~/yolo_deepstream/deepstream_yolo# git diff
diff --git a/deepstream_yolo/deepstream_app_config_yolo.txt b/deepstream_yolo/deepstream_app_config_yolo.txt
index 3ae5dee…986c6b3 100644
— a/deepstream_yolo/deepstream_app_config_yolo.txt
+++ b/deepstream_yolo/deepstream_app_config_yolo.txt
@@ -113,8 +113,8 @@ bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
nvbuf-memory-type=0
-config-file=config_infer_primary_yoloV4.txt
-#config-file=config_infer_primary_yoloV7.txt
+#config-file=config_infer_primary_yoloV4.txt
+config-file=config_infer_primary_yoloV7.txt

[tracker]
enable=0

anmolprakashnitc · March 16, 2023, 8:16am

Running yolov7 with deepstream-app tool is fine.
How can i integrate yolov7 in python apps?

mchi · March 21, 2023, 2:39pm

By below changes to “deepstream_python_apps/apps/deepstream-test1”, I confirmed it works with command:

# python3 deepstream_test_1.py /opt/nvidia/deepstream/deepstream-6.2/samples/streams/sample_720p.h264

The RED files in below “git status” are:

dstest1_pgie_config.txt → from config_infer_primary_yoloV7.txt under yolo_deepstream/deepstream_yolo at main · NVIDIA-AI-IOT/yolo_deepstream · GitHub
nvdsinfer_custom_impl_Yolo → from nvdsinfer_custom_impl_Yolo under yolo_deepstream/deepstream_yolo at main · NVIDIA-AI-IOT/yolo_deepstream · GitHub
labels.txt → from labels.txt under yolo_deepstream/deepstream_yolo at main · NVIDIA-AI-IOT/yolo_deepstream · GitHub
yolov7.onnx → from yolov7.onnx under yolo_deepstream/yolov7_qat at main · NVIDIA-AI-IOT/yolo_deepstream · GitHub