A couple of questions regarding LPD/LPR

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

DS 7.0
dGPU (T4)

While dealing with this a couple of questions came up, for which I would kindly ask for explanations.

I’m referring to this project GitHub - NVIDIA-AI-IOT/deepstream_lpr_app: Sample app code for LPR deployment on DeepStream

I have ported it to Python and then GOLANG and it works perfectly. Great so far.

This is my pipeline (just one input, one output at a time):

uridecodebin -> nvstreammux -> nvinfer (pgie, resnet18-trafficcamnet or resnet18_trafficcamnet_pruned or yolov7-tiny) -> nvtracker -> nvinfer (yolov4_tiny_usa_deployable) -> nvinfer (us_lprnet_baseline18_deployable) -> nvdosd -> nvv4l2h264enc -> rtspclientsink
  1. What makes the difference between the PGIE used model resnet18_trafficcamnet_pruned and the “original” resnet18-trafficcamnet used in all other samples? I have compared both w.r.t. GPU performance and I see a slight advantage for the pruned model, but what exactly has been pruned and why this “extra model”?

Here my (statistically not relevant) quick measurements for average temperature, GPU load and memory load for a 4 minutes 60 fps video file input for the two models:

resnet18-trafficcamnet: Temp: 40.80 deg, GPU: 37.91 %, Mem: 17.19 %
resnet18_trafficcamnet_pruned: Temp: 37.44 deg, GPU: 37.25 %, Mem: 14.80 %
yolov7-tiny: Temp: 37.5 deg, GPU: 36.96 %, Mem: 13.17 %

So I see only marginal differences. However, the overall results seem to not confirm the documented performance results from the app website (T4: 14 inputs, batch size 14 → 447 fps). I can confirm a maximum of 150 fps so far.

  1. I was trying various variations of these models, with batch size 1 and/or fp16. While all models worked with B1, some spit warnings. My impression was, that fp16 degrees the performance parameters, while the inference results are better.

Here some warnings, with a request for comment:

a) All fp16 models warned during creation: WARNING: [TRT]: builtin_op_importers.cpp:5221: Attribute caffeSemantics not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build. Nothing bad happens, though, so what is the meaning of this warning?

b) I couldn’t manage to run a yolov4_tiny_usa_deployable.etlt_b1_gpu0_int8.engine. I could create the engine, but during runtime it “crashed” like so:

:00:41.522484262 62627 0x72fc300029e0 WARN                 nvinfer gstnvinfer.cpp:1477:convert_batch_and_push_to_input_thread:<nvinfer2> error: NvBufSurfTransform failed with error -3 while converting buffer
0:00:41.522553255 62627 0x72fc300029e0 WARN                 nvinfer gstnvinfer.cpp:2420:gst_nvinfer_output_loop:<nvinfer1> error: Internal data stream error.
0:00:41.522579510 62627 0x72fc300029e0 WARN                 nvinfer gstnvinfer.cpp:2420:gst_nvinfer_output_loop:<nvinfer1> error: streaming stopped, reason error (-5)
ERRO[2024-05-20T05:54:22Z] Error: [nvinfer2] ERROR - NvBufSurfTransform failed with error -3 while converting buffer

nvinfer2 is the first SGIE inference component. What does this warning (in fact error) mean?

The general questions behind are: Is it understandable, that an FP16 engine has higher performance costs compared to an INT8? And does it make sense at all to reduce the engine to the maximum number of expected inputs at all?

  1. A third problem which I also observed with the original C app: Metadata added by nvtracker is lost if the probe is taken at the sink pad of the final sink. This is done like so in the original app. Unfortunately things like frame_meta.frame_num and frame_meta.ntp_timestamp are zeroed at this point in the pipeline. Also all tracker confidence and tracker bounding box info is 0. If the probe is taken from the source pad of nvtracker all is there.

That’s for now. Again, it is working and shows nice results.

Thanks in advance

3

You can refer to How to use this model.

Some of it may be hardcode in the open source, you can look at the code logic like deepstream_lpr_app.c

It’s just warning message, you can ignore it.

You can try to set the parameters by referring to the FAQ.

Yes.

What do you exactly mean by that?

Could you attach how you confirm that? Thanks

4

What do you exactly mean by that?

I meant: I’m only using one input. Would it make sense to downscale all used models to batch size 1 in order to gain something?

Could you attach how you confirm that? Thanks

Add before this line deepstream_lpr_app/deepstream-lpr-app/deepstream_lpr_app.c at c6c1cbcaeae423ba75c5b8b7668bada6dbf1b15c · NVIDIA-AI-IOT/deepstream_lpr_app · GitHub

      g_print("Frame number: %d, Timestamp: %ld\n", frame_meta->frame_num, frame_meta->ntp_timestamp);

You will find it all 0. I have checked the proper availability of these info elements in Python by adding a probe to nvtrackers src pad.

5

I would like to add another question:

I think this illustration is a bit misleading:

image
image906×310 52.4 KB

(from Creating a Real-Time License Plate Detection and Recognition App | NVIDIA Technical Blog)

In fact this plate would be detected as NV12345. Note the missing “spaces”. This might not be an issue for US plates, but it is for EU.

I tried the model with an EU (German) video feed. I was surprised, that it perfectly detects EU number plates, but it has a big problem: The loss of the “spaces” causes ambiguities.

For instance: A Berlin number plate can look like so: “B NO 1234”. A Bonn number plate could look like so “BN O 1234”. Due to the loss of “spaces” both are indicated as “BNO1234” which is ambiguous.

6

We recommend that streammux be configured according to the number of sources, others can be configured according to your model requirements.

After tiler plugin, we will remove all the metadata of the frame. So you cannot get correct value from the metadata after that plugin

Our current model can only support cn and us, and other scenarois need to train new models by yourself.

7

I’m not using tiler and noticed the same los. See my pipeline description. What component is removing the metadata added by tracker?

8

There’s something wrong with the piepline you posted. The nvdsosd plugin cannot support batched data. So you cannot link the nvinfer to the nvdsosd plugin. There is no problem if you have only one source. But if you want to use multiple sources, the pipeline cannot work normally.

9

There’s something wrong with the piepline you posted. The nvdsosd plugin cannot support batched data. So you cannot link the nvinfer to the nvdsosd plugin.

Well I can and it works. As said - I’m only having ONE source and I do not intend to have more.

Sorry, that you refuse to concentrate on the real issue: Why is the tracker added metadata lost at the end of the pipeleine? I gave you a simple change which you could easily add to your C code in order to double check~

Disregard please, no longer an issue. I finally understood the results…

10

Unfortunately this answer is also far away from hitting the point. OK, I’m obviously on my own here. Thanks anyway

What I wanted to ask is: The image suggests, that the inference/classification is able to detect spaces. It is NOT (or it is not documented, how)

11

Sorry, that you refuse to concentrate on the real issue: Why is the tracker added metadata lost at the end of the pipeline? I gave you a simple change which you could easily add to your C code in order to double check

I double checked the issue. I can see the timestamp and frame number in my GO code now, tracker bbox info and confidence is still lost. I’m sure I haven’t seen this with the C code, but you can ignore that gracefully

12

[quote=“foreverneilyoung, post:1, topic:293424”]
b) I couldn’t manage to run a yolov4_tiny_usa_deployable.etlt_b1_gpu0_int8.engine. I could create the engine, but during runtime it “crashed” like so:

:00:41.522484262 62627 0x72fc300029e0 WARN                 nvinfer gstnvinfer.cpp:1477:convert_batch_and_push_to_input_thread:<nvinfer2> error: NvBufSurfTransform failed with error -3 while converting buffer
0:00:41.522553255 62627 0x72fc300029e0 WARN                 nvinfer gstnvinfer.cpp:2420:gst_nvinfer_output_loop:<nvinfer1> error: Internal data stream error.
0:00:41.522579510 62627 0x72fc300029e0 WARN                 nvinfer gstnvinfer.cpp:2420:gst_nvinfer_output_loop:<nvinfer1> error: streaming stopped, reason error (-5)
ERRO[2024-05-20T05:54:22Z] Error: [nvinfer2] ERROR - NvBufSurfTransform failed with error -3 while converting buffer

Did you solve this problem? I’m also getting the same problem for the sgie1.

13

No, didn’t solve that. Preventing that at the moment.

14

I double checked the issue. I can see the timestamp and frame number in my GO code now, tracker bbox info and confidence is still lost. I’m sure I haven’t seen this with the C code, but you can ignore that gracefully

OK, in order to close this issue (which is not an issue): The tracker info is not available in all objectMeta objects coming from other entities than PGIE (which is understandable, since there is no tracker behind SGIE1 or SGIE2. Q: Would it make sense to add an nvtracker to the license plate detector?).

Maybe just one could check this:

The (partial) code for this “walk the objects” in GO: Probe taken at the sink of nvdosd, shortly before it is re-encoded and re-transmitted:

	sinkPad.AddProbe(gst.PadProbeTypeBuffer, func(pad *gst.Pad, info *gst.PadProbeInfo) gst.PadProbeReturn {
		buffer := info.GetBuffer()
		var buf *C.GstBuffer = (*C.GstBuffer)(unsafe.Pointer(buffer.Instance()))
		batchMeta := wrapNvDsBatchMeta(C.gst_buffer_get_nvds_batch_meta(buf))
		frameMetaList := batchMeta.frameMetaList()
		if frameMetaList == nil {
			return gst.PadProbeOK
		}

		for _, frameMeta := range frameMetaList {
			timestamp := frameMeta.timestamp()
			objMetaList := frameMeta.objMetaList()
			if objMetaList == nil {
				return gst.PadProbeOK
			}

			frame_number := frameMeta.frameNumber()
			timestampString := createRFC3330Timestamp(timestamp)

			log.Info(fmt.Sprintf("FrameMeta: timestamp %s, frame_nr %d, nr objects: %d", timestampString, frame_number, len(objMetaList)))

			for _, objMeta := range objMetaList {

				log.Info(fmt.Sprintf("  => ObjectMeta: unique_component_id: %d, class_id: %d, object_id: %d, detector_confidence: %.2f, detector_bbox: [%.0f, %.0f, %.0f, %.0f], tracker_confidence: %.2f, tracker_bbox: [%.0f, %.0f, %.0f, %.0f]",
					objMeta.ptr.unique_component_id,
					objMeta.ptr.class_id, 
					objMeta.ptr.object_id,
					objMeta.ptr.confidence,
					objMeta.ptr.detector_bbox_info.org_bbox_coords.left,
					objMeta.ptr.detector_bbox_info.org_bbox_coords.top,
					objMeta.ptr.detector_bbox_info.org_bbox_coords.width,
					objMeta.ptr.detector_bbox_info.org_bbox_coords.height,
					objMeta.ptr.tracker_confidence,
					objMeta.ptr.tracker_bbox_info.org_bbox_coords.left,
					objMeta.ptr.tracker_bbox_info.org_bbox_coords.top,
					objMeta.ptr.tracker_bbox_info.org_bbox_coords.width,
					objMeta.ptr.tracker_bbox_info.org_bbox_coords.height,
				))

				classifierMetaList := objMeta.classifierMetaList()
				if classifierMetaList == nil {
					continue
				}

				for _, classifierMeta := range classifierMetaList {

					infoLabelList := classifierMeta.labelInfoList()
					if infoLabelList == nil {
						continue
					}

					for _, labelInfo := range infoLabelList {
						log.Info(fmt.Sprintf("    =>  ClassifierMeta: num_labels: %d, unique_component_id: %d, result_label: %s", classifierMeta.ptr.num_labels, classifierMeta.ptr.unique_component_id, C.GoString((*C.char)(unsafe.Pointer(&labelInfo.ptr.result_label[0])))))
					}
				}

   ....

I’m first tracing FrameMeta, then - with an indention - ObjMeta then - with another indention (if available) ClassifierMeta. PGIE has unique_component_id 1, the license plate detector (SGIE1) 2 and the license plate recognizer (SGIE2) has 3. I’m observing a Berlin street scene video and wondering about the many incarnations of the Berlin number plate “B RD 1891”. I’m completely fine with all the different recognitions, I just want to understand what I get. I’m wondering about the strange, generic looking “object_id” of the LPD, but maybe this ID defines a license plate :) Or it is just some -1 :)

I would be happy, if the detector_confidence of LDP could be taken in order to judge the confidence of LPR, since there seems to be no appropriate confidence information in the ClassifierMeta…

INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.296367Z, frame_nr 142, nr objects: 3 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.99, detector_bbox: [1149, 392, 80, 25], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: BRD1891 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.60, detector_bbox: [216, 263, 86, 196], tracker_confidence: 0.46, tracker_bbox: [221, 266, 83, 195] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.53, detector_bbox: [1058, 273, 221, 236], tracker_confidence: 0.75, tracker_bbox: [1054, 273, 243, 239] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.32965Z, frame_nr 143, nr objects: 3 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.99, detector_bbox: [1154, 390, 80, 20], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 8RD18V1 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.80, detector_bbox: [223, 260, 78, 202], tracker_confidence: 0.45, tracker_bbox: [218, 263, 81, 197] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.51, detector_bbox: [1071, 269, 208, 237], tracker_confidence: 0.65, tracker_bbox: [1063, 269, 243, 238] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.362985Z, frame_nr 144, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.99, detector_bbox: [1162, 388, 79, 20], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 8RD1891 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.61, detector_bbox: [207, 258, 90, 209], tracker_confidence: 0.49, tracker_bbox: [211, 260, 84, 201] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.30, detector_bbox: [226, 352, 76, 129], tracker_confidence: 0.26, tracker_bbox: [220, 353, 88, 137] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.52, detector_bbox: [1075, 269, 203, 233], tracker_confidence: 0.64, tracker_bbox: [1070, 268, 243, 236] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.396335Z, frame_nr 145, nr objects: 3 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.98, detector_bbox: [1170, 385, 78, 19], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: GYRD1891 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.58, detector_bbox: [203, 257, 89, 204], tracker_confidence: 0.48, tracker_bbox: [204, 259, 86, 202] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.44, detector_bbox: [1086, 266, 193, 229], tracker_confidence: 0.66, tracker_bbox: [1078, 265, 243, 234] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.429745Z, frame_nr 146, nr objects: 3 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.99, detector_bbox: [1182, 381, 70, 20], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 8RD1891 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.72, detector_bbox: [196, 250, 83, 218], tracker_confidence: 0.42, tracker_bbox: [197, 255, 85, 208] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.58, detector_bbox: [1093, 240, 185, 252], tracker_confidence: 0.63, tracker_bbox: [1086, 252, 243, 240] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.463069Z, frame_nr 147, nr objects: 3 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.97, detector_bbox: [1180, 374, 80, 22], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 8MRD1891 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.59, detector_bbox: [180, 248, 93, 217], tracker_confidence: 0.45, tracker_bbox: [186, 253, 88, 211] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.56, detector_bbox: [1100, 232, 180, 255], tracker_confidence: 0.61, tracker_bbox: [1094, 241, 243, 245] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.496354Z, frame_nr 148, nr objects: 3 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.97, detector_bbox: [1191, 374, 73, 21], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: BMRD1891 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.76, detector_bbox: [182, 246, 75, 216], tracker_confidence: 0.49, tracker_bbox: [180, 250, 83, 213] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.54, detector_bbox: [1104, 228, 174, 255], tracker_confidence: 0.50, tracker_bbox: [1099, 234, 243, 249] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.529734Z, frame_nr 149, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.43, detector_bbox: [1263, 230, 16, 18], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: HW7 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.99, detector_bbox: [1195, 366, 75, 21], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 8RD1891 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.68, detector_bbox: [187, 241, 69, 226], tracker_confidence: 0.42, tracker_bbox: [179, 246, 78, 217] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.49, detector_bbox: [1113, 207, 166, 275], tracker_confidence: 0.42, tracker_bbox: [1107, 221, 243, 258] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.563134Z, frame_nr 150, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.99, detector_bbox: [1200, 363, 75, 21], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 8RD1891 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.48, detector_bbox: [152, 241, 91, 218], tracker_confidence: 0.36, tracker_bbox: [162, 243, 83, 217] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.32, detector_bbox: [159, 341, 89, 128], tracker_confidence: 0.27, tracker_bbox: [168, 353, 88, 133] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.57, detector_bbox: [1120, 222, 159, 252], tracker_confidence: 0.36, tracker_bbox: [1114, 219, 243, 256] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.596415Z, frame_nr 151, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.98, detector_bbox: [1201, 362, 74, 21], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 0TRD1891 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.65, detector_bbox: [136, 236, 85, 209], tracker_confidence: 0.46, tracker_bbox: [146, 238, 83, 215] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.40, detector_bbox: [165, 342, 78, 126], tracker_confidence: 0.29, tracker_bbox: [163, 348, 84, 130] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.35, detector_bbox: [1127, 218, 151, 251], tracker_confidence: 0.41, tracker_bbox: [1121, 216, 243, 254] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.629708Z, frame_nr 152, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.98, detector_bbox: [1209, 359, 66, 20], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 8RD189 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.67, detector_bbox: [129, 237, 98, 212], tracker_confidence: 0.44, tracker_bbox: [134, 237, 88, 214] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.41, detector_bbox: [153, 329, 77, 136], tracker_confidence: 0.40, tracker_bbox: [154, 340, 82, 132] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.38, detector_bbox: [1128, 220, 152, 247], tracker_confidence: 0.39, tracker_bbox: [1126, 216, 243, 252] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.663101Z, frame_nr 153, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.97, detector_bbox: [1213, 354, 64, 21], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 8RD189 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.54, detector_bbox: [125, 233, 91, 217], tracker_confidence: 0.45, tracker_bbox: [126, 235, 89, 215] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.32, detector_bbox: [143, 330, 78, 133], tracker_confidence: 0.41, tracker_bbox: [145, 335, 81, 133] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 15, detector_confidence: 0.48, detector_bbox: [1135, 201, 144, 263], tracker_confidence: 0.37, tracker_bbox: [1132, 207, 243, 256] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.696435Z, frame_nr 154, nr objects: 2 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.51, detector_bbox: [105, 231, 101, 212], tracker_confidence: 0.53, tracker_bbox: [113, 232, 93, 214] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.62, detector_bbox: [100, 307, 113, 152], tracker_confidence: 0.40, tracker_bbox: [122, 323, 92, 139] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.730364Z, frame_nr 155, nr objects: 1 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.54, detector_bbox: [98, 227, 98, 219], tracker_confidence: 0.43, tracker_bbox: [102, 229, 95, 216] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.763747Z, frame_nr 156, nr objects: 1 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.48, detector_bbox: [96, 228, 87, 215], tracker_confidence: 0.39, tracker_bbox: [95, 228, 92, 215] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.796364Z, frame_nr 157, nr objects: 1 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.59, detector_bbox: [90, 227, 86, 223], tracker_confidence: 0.43, tracker_bbox: [89, 228, 90, 218] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.829646Z, frame_nr 158, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.95, detector_bbox: [1234, 347, 45, 18], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: BR01 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.54, detector_bbox: [71, 223, 92, 219], tracker_confidence: 0.46, tracker_bbox: [77, 225, 91, 218] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.32, detector_bbox: [92, 324, 80, 137], tracker_confidence: 0.23, tracker_bbox: [93, 323, 87, 138] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 16, detector_confidence: 0.43, detector_bbox: [1157, 264, 121, 183], tracker_confidence: 0.62, tracker_bbox: [1152, 249, 138, 203] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.863014Z, frame_nr 159, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.97, detector_bbox: [1238, 346, 40, 17], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.53, detector_bbox: [59, 223, 92, 218], tracker_confidence: 0.41, tracker_bbox: [64, 224, 91, 218] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.59, detector_bbox: [65, 311, 96, 146], tracker_confidence: 0.27, tracker_bbox: [76, 319, 90, 141] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 16, detector_confidence: 0.38, detector_bbox: [1165, 263, 114, 180], tracker_confidence: 0.53, tracker_bbox: [1159, 253, 138, 195] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.89635Z, frame_nr 160, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.96, detector_bbox: [1244, 345, 34, 16], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.55, detector_bbox: [42, 228, 101, 213], tracker_confidence: 0.44, tracker_bbox: [50, 225, 94, 216] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.70, detector_bbox: [39, 318, 114, 140], tracker_confidence: 0.26, tracker_bbox: [55, 318, 98, 141] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 16, detector_confidence: 0.48, detector_bbox: [1169, 239, 110, 203], tracker_confidence: 0.37, tracker_bbox: [1162, 245, 138, 198] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.929624Z, frame_nr 161, nr objects: 4 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.98, detector_bbox: [1244, 345, 35, 17], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: EAHPRR0 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.46, detector_bbox: [29, 223, 97, 220], tracker_confidence: 0.47, tracker_bbox: [35, 224, 95, 218] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.47, detector_bbox: [47, 321, 92, 139], tracker_confidence: 0.25, tracker_bbox: [44, 319, 96, 140] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 16, detector_confidence: 0.56, detector_bbox: [1172, 245, 107, 195], tracker_confidence: 0.49, tracker_bbox: [1167, 245, 138, 197] 
INFO[2024-05-21T08:49:20Z] FrameMeta: timestamp 2024-05-21T08:49:20.963018Z, frame_nr 162, nr objects: 5 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.93, detector_bbox: [1247, 345, 33, 16], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:20Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: ELJR8 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 17, detector_confidence: 0.24, detector_bbox: [885, 264, 28, 101], tracker_confidence: 0.81, tracker_bbox: [886, 262, 29, 102] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.62, detector_bbox: [21, 216, 103, 220], tracker_confidence: 0.48, tracker_bbox: [24, 220, 98, 219] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.80, detector_bbox: [19, 321, 117, 140], tracker_confidence: 0.23, tracker_bbox: [27, 319, 103, 140] 
INFO[2024-05-21T08:49:20Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 16, detector_confidence: 0.57, detector_bbox: [1173, 257, 107, 183], tracker_confidence: 0.52, tracker_bbox: [1170, 250, 138, 192] 
INFO[2024-05-21T08:49:21Z] FrameMeta: timestamp 2024-05-21T08:49:20.996335Z, frame_nr 163, nr objects: 4 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.93, detector_bbox: [1249, 345, 31, 17], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.62, detector_bbox: [7, 228, 98, 209], tracker_confidence: 0.43, tracker_bbox: [11, 223, 98, 215] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.56, detector_bbox: [12, 320, 108, 141], tracker_confidence: 0.22, tracker_bbox: [15, 319, 105, 140] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 16, detector_confidence: 0.59, detector_bbox: [1174, 257, 106, 183], tracker_confidence: 0.38, tracker_bbox: [1172, 253, 138, 189] 
INFO[2024-05-21T08:49:21Z] FrameMeta: timestamp 2024-05-21T08:49:21.029692Z, frame_nr 164, nr objects: 4 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.85, detector_bbox: [1249, 345, 30, 16], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.68, detector_bbox: [0, 233, 99, 206], tracker_confidence: 0.43, tracker_bbox: [2, 227, 98, 212] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.47, detector_bbox: [0, 310, 108, 156], tracker_confidence: 0.28, tracker_bbox: [3, 315, 106, 146] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 16, detector_confidence: 0.57, detector_bbox: [1177, 261, 103, 179], tracker_confidence: 0.53, tracker_bbox: [1175, 256, 138, 185] 
INFO[2024-05-21T08:49:21Z] FrameMeta: timestamp 2024-05-21T08:49:21.063074Z, frame_nr 165, nr objects: 5 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 2, class_id: 0, object_id: 18446744073709551615, detector_confidence: 0.96, detector_bbox: [1252, 345, 27, 17], tracker_confidence: 0.00, tracker_bbox: [0, 0, 0, 0] 
INFO[2024-05-21T08:49:21Z]     =>  ClassifierMeta: num_labels: 1, unique_component_id: 3, result_label: 7718 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 17, detector_confidence: 0.53, detector_bbox: [871, 266, 29, 105], tracker_confidence: 0.66, tracker_bbox: [872, 265, 29, 104] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.69, detector_bbox: [0, 235, 84, 200], tracker_confidence: 0.39, tracker_bbox: [-9, 232, 98, 208] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.57, detector_bbox: [0, 311, 96, 155], tracker_confidence: 0.33, tracker_bbox: [-7, 314, 106, 149] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 0, object_id: 16, detector_confidence: 0.58, detector_bbox: [1180, 265, 99, 175], tracker_confidence: 0.40, tracker_bbox: [1178, 260, 138, 182] 
INFO[2024-05-21T08:49:21Z] FrameMeta: timestamp 2024-05-21T08:49:21.096338Z, frame_nr 166, nr objects: 3 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 17, detector_confidence: 0.29, detector_bbox: [868, 267, 27, 106], tracker_confidence: 0.73, tracker_bbox: [868, 266, 28, 105] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 2, object_id: 13, detector_confidence: 0.51, detector_bbox: [0, 237, 74, 210], tracker_confidence: 0.43, tracker_bbox: [-19, 235, 98, 209] 
INFO[2024-05-21T08:49:21Z]   => ObjectMeta: unique_component_id: 1, class_id: 1, object_id: 14, detector_confidence: 0.32, detector_bbox: [6, 331, 79, 135], tracker_confidence: 0.33, tracker_bbox: [-16, 321, 106, 144]

Makes sense?

15

Let’s solve the problem as follows.

  1. Could you attach the video that with blank in the license?
  2. Let’s tune the C version first. After the C version is OK, you can change to another program language.
16
  1. Could you attach the video that with blank in the license?

Hmm. Not publicly. In fact I made the video yesterday, but here in Germany privacy is a high value. Can I forward you the link to the video on Dropbox privately?

17

OK. You can also click my icon and message to me directly.

18

The blog may be a little misleading. You can refer to the description, here’s a picture of our inferred results. As I attached before, the model we are currently using only supports license plates of US and Chinese. If you want to support other special license plates, you need to train the model yourself.

19

The blog may be a little misleading. You can refer to the description, here’s a picture of our inferred results. As I attached before, the model we are currently using only supports license plates of US and Chinese. If you want to support other special license plates, you need to train the model yourself.

I totally understand that. OK, no issue.

20

I would try to learn how to do transfer learning and re-training of these models, but the topic is very weird and complex.

I started here Creating a Real-Time License Plate Detection and Recognition App | NVIDIA Technical Blog but any link I click, for instance this one NVIDIA NGC leads me out and finally ends up here

image
image914×394 4.81 KB

What to do to get started with this?

22

Please find it in this ngc link https://catalog.ngc.nvidia.com/orgs/nvidia/teams/tao/models/lpdnet.