Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) GPU • DeepStream Version 6.2 • JetPack Version (valid for Jetson only) • TensorRT Version 8.5.2.2
I’m developing a real-time multi-stream Python application that consists of a PGIE (TrafficCamNet), and two SGIE’s:LPD and LPR.
The LPR model is having difficulties to recognize the characters of the license plate (I guess that’s due to their resolution) and I want to do an upsampling for each of the crops detected by the LPD model and pass them to the LPR model. What is the best approach without degrading the performance in terms of FPS? And how can I do it?
As you can see in the images everything is working correctly but the LPRNet is unable to recognize the license plate (for videos with higher resolution it works quite well).
Are you using our pre-trained LPR model or the model you trained by yourself? Have you tested whether the upscaling method will improve the recognition result with some simple experiments?
The gst-nvinfer and nvinfer library are open source, you need to implement your upsampling algorithm in CUDA and add it to the pre-processing part of gst-nvinfer. This may reduce the impact to the performance as much as possible.
Yes, we tested with the pre-trained LPR model and also with one fine-tuned it with our data but in both cases we are missing to recognize license plates. We wanted to upscale the images of the bounding boxes and then feed it into LPRnet but we don’t know how to do it that’s why we were unable to test it. Could you provide us with more information regarding how to apply the upsampling algorithm or provide us with some examples (in Python, if exist)
Please find and implement the upsampling(interpolation) algorithms by yourself.
Personally I don’t think the ordinary upsampling/interpolation (E.G. nearest, bilinear, cubic,… , these algorithms are used in tf.keras.layers.UpSampling2D | TensorFlow v2.12.0) of the image will help your case. The gst-nvinfer already implements these algorithm inside to scale the small images to the model input resolution for inferencing, the accuracy does not satisfy you.
Ok, thank you for your answer. Then what could be the reason why in this situation we are not able to recognize any of the license plate characters? Whenever the LPs are bigger it is able to give us an output.
I don’t think that’s the issue, we already retrained model with such pictures and the result was the same. We tested the same video by increasing the resolution from 1280 × 720 to 1920 x 1080 and the license plates were recognized so the size of the license plate has an impact to the performance of the model even if the quality of the video is the same. How we should apply upsampling, even if it is in the whole video not only the output of sgie? Is there any preprocess plugin or some configuration on any of the deepstream elements that can apply upsampling to the input stream?
