hi:
Hello, I have referencedhttps://devtalk.nvidia.com/default/topic/1067568/deepstream-sdk/detector1-gt-cropped-images-gt-detector-2/#,The instance model is a caffe model. I only have a yolo auxiliary detector. I use the primary_detector_config.txt original file as the primary detector,My secondary detector is as follows:
[property]
gpu-id=0
net-scale-factor=1
#0=RGB, 1=BGR
model-color-format=0
custom-network-config=yolov3-tiny.cfg
model-file=yolov3-tiny.weights
#model-engine-file=model_b1_fp32.engine
labelfile-path=labels.txt
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
num-detected-classes=80
gie-unique-id=1
is-classifier=0
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomYoloV3Tiny
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
报错:
root@uu-desktop:/opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/back-to-back-detectors# ./back-to-back-detectors ../../../../samples/streams/test.mp4
Now playing: ../../../../samples/streams/test.mp4
Using winsys: x11
Opening in BLOCKING MODE
Creating LL OSD context new
0:00:00.653026063 870 0x5586c82870 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary-nvinference-engine2> NvDsInferContext[UID 2]:initialize(): Trying to create engine from model files
Loading pre-trained weights...
Loading complete!
Total Number of weights read : 8858734
layer inp_size out_size weightPtr
(1) conv-bn-leaky 3 x 416 x 416 16 x 416 x 416 496
(2) maxpool 16 x 416 x 416 16 x 208 x 208 496
(3) conv-bn-leaky 16 x 208 x 208 32 x 208 x 208 5232
(4) maxpool 32 x 208 x 208 32 x 104 x 104 5232
(5) conv-bn-leaky 32 x 104 x 104 64 x 104 x 104 23920
(6) maxpool 64 x 104 x 104 64 x 52 x 52 23920
(7) conv-bn-leaky 64 x 52 x 52 128 x 52 x 52 98160
(8) maxpool 128 x 52 x 52 128 x 26 x 26 98160
(9) conv-bn-leaky 128 x 26 x 26 256 x 26 x 26 394096
(10) maxpool 256 x 26 x 26 256 x 13 x 13 394096
(11) conv-bn-leaky 256 x 13 x 13 512 x 13 x 13 1575792
(12) maxpool 512 x 13 x 13 512 x 13 x 13 1575792
(13) conv-bn-leaky 512 x 13 x 13 1024 x 13 x 13 6298480
(14) conv-bn-leaky 1024 x 13 x 13 256 x 13 x 13 6561648
(15) conv-bn-leaky 256 x 13 x 13 512 x 13 x 13 7743344
(16) conv-linear 512 x 13 x 13 255 x 13 x 13 7874159
(17) yolo 255 x 13 x 13 255 x 13 x 13 7874159
(18) route - 256 x 13 x 13 7874159
(19) conv-bn-leaky 256 x 13 x 13 128 x 13 x 13 7907439
(20) upsample 128 x 13 x 13 128 x 26 x 26 -
(21) route - 384 x 26 x 26 7907439
(22) conv-bn-leaky 384 x 26 x 26 256 x 26 x 26 8793199
(23) conv-linear 256 x 26 x 26 255 x 26 x 26 8858734
(24) yolo 255 x 26 x 26 255 x 26 x 26 8858734
Output blob names :
yolo_17
yolo_24
Total number of layers: 50
Total number of layers on DLA: 0
Building the TensorRT Engine...
Building complete!
0:00:28.364973192 870 0x5586c82870 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary-nvinference-engine2> NvDsInferContext[UID 2]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/back-to-back-detectors/model_b1_fp32.engine
Deserialize yoloLayerV3 plugin: yolo_17
Deserialize yoloLayerV3 plugin: yolo_24
0:00:28.834257441 870 0x5586c82870 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary-nvinference-engine1> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:28.834410880 870 0x5586c82870 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary-nvinference-engine1> NvDsInferContext[UID 1]:generateTRTModel(): INT8 not supported by platform. Trying FP16 mode.
0:01:07.114985055 870 0x5586c82870 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary-nvinference-engine1> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_fp16.engine
Running...
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Creating LL OSD context new
Frame Number = 0 Vehicle Count = 6 Person Count = 6 Face Count = 0 License Plate Count = 0
0:01:08.032555122 870 0x5586c79f20 WARN nvinfer gstnvinfer.cpp:1149:convert_batch_and_push_to_input_thread:<primary-nvinference-engine2> error: NvBufSurfTransform failed with error -2 while converting buffer
0:01:08.032898928 870 0x5586c79f20 WARN nvinfer gstnvinfer.cpp:1830:gst_nvinfer_output_loop:<primary-nvinference-engine1> error: Internal data stream error.
0:01:08.033508908 870 0x5586c79f20 WARN nvinfer gstnvinfer.cpp:1830:gst_nvinfer_output_loop:<primary-nvinference-engine1> error: streaming stopped, reason error (-5)
ERROR from element primary-nvinference-engine2: NvBufSurfTransform failed with error -2 while converting buffer
Error details: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(1149): convert_batch_and_push_to_input_thread (): /GstPipeline:pipeline/GstNvInfer:primary-nvinference-engine2
Returned, stopping playback
0:01:08.052820395 870 0x5586c79f20 WARN nvinfer gstnvinfer.cpp:1149:convert_batch_and_push_to_input_thread:<primary-nvinference-engine2> error: NvBufSurfTransform failed with error -3 while converting buffer
double free or corruption (out)
Aborted (core dumped)
Hi,
It looks like some issue in the buffer:
ERROR from element primary-nvinference-engine2: NvBufSurfTransform failed with error -2 while converting buffer
Error details: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(1149): convert_batch_and_push_to_input_thread (): /GstPipeline:pipeline/GstNvInfer:primary-nvinference-engine2
Returned, stopping playback
Could you try to add ‘process-mode=2’ configure in your YOLO detector and try it again?
[property]
gpu-id=0
<b>process-mode=2</b>
...
Thanks.
hi AstaLLL:
The problem is not solved,
root@uu-desktop:/opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/back-to-back-detectors# ./back-to-back-detectors ../../../../samples/streams/test.mp4
Now playing: ../../../../samples/streams/test.mp4
Using winsys: x11
Opening in BLOCKING MODE
Creating LL OSD context new
0:00:00.714180655 3082 0x55a5182670 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary-nvinference-engine2> NvDsInferContext[UID 2]:initialize(): Trying to create engine from model files
Loading pre-trained weights...
Loading complete!
Total Number of weights read : 8858734
layer inp_size out_size weightPtr
(1) conv-bn-leaky 3 x 416 x 416 16 x 416 x 416 496
(2) maxpool 16 x 416 x 416 16 x 208 x 208 496
(3) conv-bn-leaky 16 x 208 x 208 32 x 208 x 208 5232
(4) maxpool 32 x 208 x 208 32 x 104 x 104 5232
(5) conv-bn-leaky 32 x 104 x 104 64 x 104 x 104 23920
(6) maxpool 64 x 104 x 104 64 x 52 x 52 23920
(7) conv-bn-leaky 64 x 52 x 52 128 x 52 x 52 98160
(8) maxpool 128 x 52 x 52 128 x 26 x 26 98160
(9) conv-bn-leaky 128 x 26 x 26 256 x 26 x 26 394096
(10) maxpool 256 x 26 x 26 256 x 13 x 13 394096
(11) conv-bn-leaky 256 x 13 x 13 512 x 13 x 13 1575792
(12) maxpool 512 x 13 x 13 512 x 13 x 13 1575792
(13) conv-bn-leaky 512 x 13 x 13 1024 x 13 x 13 6298480
(14) conv-bn-leaky 1024 x 13 x 13 256 x 13 x 13 6561648
(15) conv-bn-leaky 256 x 13 x 13 512 x 13 x 13 7743344
(16) conv-linear 512 x 13 x 13 255 x 13 x 13 7874159
(17) yolo 255 x 13 x 13 255 x 13 x 13 7874159
(18) route - 256 x 13 x 13 7874159
(19) conv-bn-leaky 256 x 13 x 13 128 x 13 x 13 7907439
(20) upsample 128 x 13 x 13 128 x 26 x 26 -
(21) route - 384 x 26 x 26 7907439
(22) conv-bn-leaky 384 x 26 x 26 256 x 26 x 26 8793199
(23) conv-linear 256 x 26 x 26 255 x 26 x 26 8858734
(24) yolo 255 x 26 x 26 255 x 26 x 26 8858734
Output blob names :
yolo_17
yolo_24
Total number of layers: 50
Total number of layers on DLA: 0
Building the TensorRT Engine...
Building complete!
0:00:29.349012887 3082 0x55a5182670 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary-nvinference-engine2> NvDsInferContext[UID 2]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/back-to-back-detectors/model_b1_fp32.engine
Deserialize yoloLayerV3 plugin: yolo_17
Deserialize yoloLayerV3 plugin: yolo_24
0:00:29.808689508 3082 0x55a5182670 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary-nvinference-engine1> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:29.808830915 3082 0x55a5182670 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary-nvinference-engine1> NvDsInferContext[UID 1]:generateTRTModel(): INT8 not supported by platform. Trying FP16 mode.
0:01:08.540191801 3082 0x55a5182670 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary-nvinference-engine1> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_fp16.engine
Running...
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Creating LL OSD context new
Frame Number = 0 Vehicle Count = 6 Person Count = 6 Face Count = 0 License Plate Count = 0
0:01:09.468900354 3082 0x55a5179f20 WARN nvinfer gstnvinfer.cpp:1149:convert_batch_and_push_to_input_thread:<primary-nvinference-engine2> error: NvBufSurfTransform failed with error -2 while converting buffer
0:01:09.469822617 3082 0x55a5179f20 WARN nvinfer gstnvinfer.cpp:1830:gst_nvinfer_output_loop:<primary-nvinference-engine1> error: Internal data stream error.
ERROR from element primary-nvinference-engine2: NvBufSurfTransform failed with error -2 while converting buffer
0:01:09.469968504 3082 0x55a5179f20 WARN nvinfer gstnvinfer.cpp:1830:gst_nvinfer_output_loop:<primary-nvinference-engine1> error: streaming stopped, reason error (-5)
Error details: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(1149): convert_batch_and_push_to_input_thread (): /GstPipeline:pipeline/GstNvInfer:primary-nvinference-engine2
Returned, stopping playback
0:01:09.489323239 3082 0x55a5179f20 WARN nvinfer gstnvinfer.cpp:1149:convert_batch_and_push_to_input_thread:<primary-nvinference-engine2> error: NvBufSurfTransform failed with error -3 while converting buffer
double free or corruption (out)
Aborted (core dumped)
Does the darknet algorithm support this? Back-to-back uses the caffe model.
hi:
Is there something wrong with my model? Can you provide yolo’s algorithm, let me test it?
Hi,
Let me give it a try.
Will get back to you asap.
Thanks.
hi AastaLLL:
I am concerned that our deepstream yolo interface only has vehicle detection applications. In objectDetect_Yolo, does the nvinfer in deepstream support back-to-back applications? Have you tried it? Can our deepstream provide a set of yolo applications for deepstream-test2, not just caffee.
Thank you
hi AastaLLL:
Any test results?
Dear AastaLLL:
Can I provide you the yolo algorithm I used for testing? I can email you if needed
Hi,
Sorry to keep you waiting.
We are still trying to reproduce this issue on our side.
A simple reproducible from you will definitely help us to debugging.
You can share the source from private message.
Thanks.
Dear AastaLLL:
Can the yolo algorithm you used for testing be shared with me for reference? If the nvinfer plugin is really no problem for yolo, then I can be sure that it is a problem with my darknet algorithm. Thank you very much
Hi,
Sorry for keeping you waiting.
We can reproduce this issue with our official YOLO model directly.
This issue will be passed into our internal team and will update more information with you once we got a feedback.
Thanks.
1 Like
Hi,
We found out the cause of this issue.
This is because the YOLO network resolution is 416x416.
However, there is no limitation on the size of object crops sent for secondary inference.
Jetson scaling will fail for factor of 16.
To fix this, please set input-object-min-width = 28 and input-object-min-height = 28.
We have verified that YOLO can work correctly as secondary detector with the update.
secondary_detector_config.txt
[property]
...
input-object-min-width=28
input-object-min-height=28
Thanks.
hi AastaLLL:
Thank you for your excellent work, I can use yolo according to your method. However, it seems that there are still some problems that need to be adapted. When my deepstream runs back-to-back, the image is very stuttered, as shown below file:///C:/Users/Administrator/Documents/Tencent%20Files/374789127/FileRecv/MobileFile/IMG_3102.JPG, how do I adapt it?
yolo.zip (31.1 MB)
hi AastaLLL:
This is the yolo auxiliary detector I used. It is an algorithm to detect the lights. The algorithm input is a picture of the size of the vehicle. Please help me to see what is the reason, thank you very much!
Hi,
Sorry for the late update.
For a customized YOLO model, you will need to update several parameter based on your setting.
Please follow our tutorial to setup the parameter:
https://docs.nvidia.com/metropolis/deepstream/Custom_YOLO_Model_in_the_DeepStream_YOLO_App.pdf
Thanks.
hi AastaLLL:
Hello, I trained according to the darknet provided by you, and set the parameters according to your setup tutorial, and our same training method can be applied to vehicle detection on tx2, but auxiliary detection cannot be applied, can you help me Test it? I have provided the corresponding algorithm in the attachment.
Hi,
Sorry for the late update.
Do you mean the attachment in comment#13?
Thanks.
hi AastaLLL:
Yes, thank you for your reply and look forward to your feedback on testing.
Hi, 374789127
Sorry for the late reply.
Have you fixed this issue?
Does custom the YOLO parameter with this document helps?
Thanks.