-
[DSx_All_App] Debug Tips for DeepStream Accuracy Issue
Ensure the image pre-process before inference aligns with the training pre-process.
15.1 Confirm your model has got good accuracy in training and inference outside DeepStream
15.2 nvinfer
When deploying a ONNX model to DeepStream with nvinfer plugin, confirm below nvinfer parameters are set correctly to align with the corresponding settings in Training
15.2.1 Input scale & offset
1). net-scale-factor =
2). offsets
The usage of these two parameters are as below (from doc)
15.2.2 Input Order
1). network-input-order= // 0:NCHW 1:NHWC
2). infer-dims= // if network-input-order=1, i.e. NHWC, infer-dims must be specified, otherwise, nvinfer can’t detect input dims automatically
3). model-color-format= // 0: RGB 1: BGR 2: GRAY
15.2.3 scale and padding
1). maintain-aspect-ratio= // whether to maintain aspect ratio while scaling input
2). symmetric-padding= // whether to pad image symmetrically while scaling input. By defaulut, it’s asymmetrical padding and the image will be scaled to top left corner.
15.2.4 inference precision
1). network-mode= // 0: FP32 1: INT8 2: FP16. If INT8 accuracy is not good, try FP16 or FP32
15.2.5 threshold
1). threshold=
2). pre-cluster-threshold=
3). Post-cluster-threshold=
Above are some highlighted parameters for a quick check for accuracy. For more detailed informantion, please refer to nvinfer doc - Gst-nvinfer — DeepStream 6.2 Release documentation
15.2.6 avoid missing the object that close to the border of the image(Version 6.2 and above)
1). crop-objects-to-roi-boundary=1
15.3 Dump the input or output of the nvinfer
Below two items in DeepStream SDK FAQ - #9 by mchi
2. [DS5.0GA_Jetson_dGPU_Plugin] Dump the Inference Input ==> compare the input between DS and your own standalone inference/training app
3. [DS5_Jetson_dGPU_Plugin] Dump the Inference outputs ==> then apply your own parser offline check this output data
15.4 Try to remove/replace the plugin with extra conversion.
15.4.1 The following two pipelines have the same result before Gst-nvstreammux plugin, you can choose the second one to reduce extra conversions caused by videoconvert and Gst-nvvideoconvert.multifilesrc->jpegdec->videoconvert->nvvideoconvert->nvstreammux->nvinfer… multifilesrc->nvjpegdec->nvstreammux->nvinfer……15.4.2 If you use Gst-nvstreammux, please set the width and height paremeters to align with the video.
We suggest you try to use the Gst-nvstreammux New instead of Gst-nvstreammux especially when the width or height of video is not multiples of 4. If you need to use Gst-nvstreammux New, please ensure that all sources have the same resolution.
