How can I use nvdspreprocess before secondary nvinfer to do some perspective job?

[2624232]

Hello NVIDIA. After I get the object box and other properties(just like the 4 landmark points of a car licent plate) from the primary nvinfer, I need to do some perspective job on the object. How can I use nvdspreprocess plugin to do this job and prepare the wrapped object tensor to the lpr infer plugin? I read the deepstream-preprocess-test example code, but it seemed running only before primary nvinfer to provide the ROI tensor preperation.

[Fiona.Chen]

@2624232

After I get the object box and other properties(just like the 4 landmark points of a car licent plate) from the primary nvinfer, I need to do some perspective job on the object. How can I use nvdspreprocess plugin to do this job and prepare the wrapped object tensor to the lpr infer plugin?

What does your lpr infer plugin need?

[2624232]

I intent to use LPRNet for Chinese Plate recognition.

[Fiona.Chen]

Seems you need to implement your own “custom-input-transformation-function” of gst-nvdspreprocess. The LPRNet should be used as PGIE instead of SGIE.

[2624232]

The pipeline is:
filesrc(read mp4 file) → qtdemux → h264parse → avdec_h264 → nvvideoconvert → capsfilter → nvstreammux → pgie (detectnet_v2 for detecting the vechicle) → nvtracker → sgie(wpodnet for detecting the licence plate) → sgie (lprnet for recognizing the plate) → nvdsosd → nveglglessink
The sgie(wpodnet for detecting the licence plate) will get the bbox and 4 corner coordinates of the plate. So I want to insert a nvdspreprocess plugin after the it to do the perspective wrap acording to the 4 corners. How can I do it?

[Fiona.Chen]

@2624232

I want to insert a nvdspreprocess plugin after the it to do the perspective wrap acording to the 4 corners. How can I do it?

As I answered above. The gst-nvdspreprocess provide interface to batch the cropped and processed images, you don’t need SGIE any more. The only thing you need to do is to implement the “get the bbox and 4 corner coordinates of the plate” with “custom-input-transformation-function”.

[2624232]

I was confused with custom_transform_function and custom_tensor_function when I read the source code of nvdspreprocess_lib. Is there any examples to implement of these funcions? What’s difference of them? How can I prepare tensor for my lpr nvinfer?

[Fiona.Chen]

custom_transform_function is the preprocess algorithm implementation. The default preprocess algorithm is to crop ROI from frames and scaling to batch dimension. Th user needs to implement his own preprocess algorithm if he needs different preprocess.

custom_tensor_function is to generate customized tensor. The default implementation can generate NHWC or NCHW tensor data. If the user needs other type of tensor data, he needs to implement the tensor generation by custom_tensor_function.