How to interpret obj_meta.rect_params?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson Nano
• DeepStream Version
5.0.1
• JetPack Version (valid for Jetson only)
4.4
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hi,

I’m having some doubts regarding the interpretation of the obj_meta.rect_params info element.

I’m running the Python sample deepstream-test3 (deepstream_python_apps/apps/deepstream-test3 at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub) and using a pre-recorded MP4 file (640x480).

I literally just made marginal changes to the original code (e.g. changing the tiler output width/height, re-directing the inference engine to use FP16 and cluster-model=3). I don’t have an explanation for the discrepancy I see between the on-screen display and the reported image coordinate.

Basically I was trying to determine the image center coordinate of the bottom of the rectangle in order to determine distance and angle of the detection.

Here is a snapshot of the on-screen display (person anonymized):

image
image1272×1012 278 KB

The same time the algorithm reported the displayed coordinates. Those are nonsense, IMHO, given the size of the input.

        print("Idx: {0:d}, ID: {1:s}, Base: X: {2:d} Y: {3:d}, conf: {4:.8f}".format(
            frame_meta.pad_index,
            pgie_classes_str[obj_meta.class_id],
            round(obj_meta.rect_params.left + obj_meta.rect_params.width / 2), 
            round(obj_meta.rect_params.top + obj_meta.rect_params.height), 
            obj_meta.confidence))

Some traces of the added print:

Idx: 0, ID: Person, Base: X: 689 Y: 643, conf: 0.26855469
Idx: 0, ID: Person, Base: X: 689 Y: 642, conf: 0.26440430
Idx: 0, ID: Person, Base: X: 690 Y: 642, conf: 0.24414062
Idx: 0, ID: Person, Base: X: 688 Y: 643, conf: 0.23681641
Idx: 0, ID: Person, Base: X: 688 Y: 642, conf: 0.23352051
Idx: 0, ID: Person, Base: X: 687 Y: 642, conf: 0.20947266
Idx: 0, ID: Person, Base: X: 686 Y: 643, conf: 0.20166016
Idx: 0, ID: Person, Base: X: 686 Y: 643, conf: 0.21850586
Idx: 0, ID: Person, Base: X: 687 Y: 642, conf: 0.20214844
Idx: 0, ID: Person, Base: X: 692 Y: 642, conf: 0.21032715
Idx: 0, ID: Person, Base: X: 691 Y: 645, conf: 0.28955078
Idx: 0, ID: Person, Base: X: 691 Y: 649, conf: 0.30981445
Idx: 0, ID: Person, Base: X: 690 Y: 645, conf: 0.48681641
Idx: 0, ID: Person, Base: X: 690 Y: 648, conf: 0.53515625
Idx: 0, ID: Person, Base: X: 692 Y: 647, conf: 0.48071289
Idx: 0, ID: Person, Base: X: 693 Y: 648, conf: 0.39306641
Idx: 0, ID: Person, Base: X: 695 Y: 649, conf: 0.44384766
Idx: 0, ID: Person, Base: X: 696 Y: 648, conf: 0.35131836
Idx: 0, ID: Person, Base: X: 695 Y: 647, conf: 0.34130859
Idx: 0, ID: Person, Base: X: 696 Y: 648, conf: 0.32568359
Idx: 0, ID: Person, Base: X: 697 Y: 652, conf: 0.42578125
Idx: 0, ID: Person, Base: X: 697 Y: 649, conf: 0.45458984

Is there some scale factor to be recognized and used?

This is the patch for deepstream_test_3.py:

diff --git a/apps/deepstream-test3/deepstream_test_3.py b/apps/deepstream-test3/deepstream_test_3.py
index 4b4fddb..42b1413 100644
--- a/apps/deepstream-test3/deepstream_test_3.py
+++ b/apps/deepstream-test3/deepstream_test_3.py
@@ -50,8 +50,8 @@ PGIE_CLASS_ID_ROADSIGN = 3
 MUXER_OUTPUT_WIDTH=1920
 MUXER_OUTPUT_HEIGHT=1080
 MUXER_BATCH_TIMEOUT_USEC=4000000
-TILED_OUTPUT_WIDTH=1280
-TILED_OUTPUT_HEIGHT=720
+TILED_OUTPUT_WIDTH=640
+TILED_OUTPUT_HEIGHT=480
 GST_CAPS_FEATURES_NVMM="memory:NVMM"
 OSD_PROCESS_MODE= 0
 OSD_DISPLAY_TEXT= 0
@@ -107,6 +107,17 @@ def tiler_src_pad_buffer_probe(pad,info,u_data):
             except StopIteration:
                 break
             obj_counter[obj_meta.class_id] += 1
+
+
+            print("Idx: {0:d}, ID: {1:s}, Base: X: {2:d} Y: {3:d}, conf: {4:.8f}".format(
+                frame_meta.pad_index,
+                pgie_classes_str[obj_meta.class_id],
+                round(obj_meta.rect_params.left + obj_meta.rect_params.width / 2), 
+                round(obj_meta.rect_params.top + obj_meta.rect_params.height), 
+                obj_meta.confidence))
+
+
+
             try: 
                 l_obj=l_obj.next
             except StopIteration:
@@ -130,7 +141,7 @@ def tiler_src_pad_buffer_probe(pad,info,u_data):
         py_nvosd_text_params.text_bg_clr.alpha = 1.0
         #print("Frame Number=", frame_number, "Number of Objects=",num_rects,"Vehicle_count=",vehicle_count,"Person_count=",person)
         pyds.nvds_add_display_meta_to_frame(frame_meta, display_meta)"""
-        print("Frame Number=", frame_number, "Number of Objects=",num_rects,"Vehicle_count=",obj_counter[PGIE_CLASS_ID_VEHICLE],"Person_count=",obj_counter[PGIE_CLASS_ID_PERSON])
+        #print("Frame Number=", frame_number, "Number of Objects=",num_rects,"Vehicle_count=",obj_counter[PGIE_CLASS_ID_VEHICLE],"Person_count=",obj_counter[PGIE_CLASS_ID_PERSON])
 
         # Get frame rate through this probe
         fps_streams["stream{0}".format(frame_meta.pad_index)].get_fps()

And this the patch for dstest3_pgie_config.txt:

diff --git a/apps/deepstream-test3/dstest3_pgie_config.txt b/apps/deepstream-test3/dstest3_pgie_config.txt
index a6a797f..301b565 100644
--- a/apps/deepstream-test3/dstest3_pgie_config.txt
+++ b/apps/deepstream-test3/dstest3_pgie_config.txt
@@ -60,11 +60,13 @@
 [property]
 gpu-id=0
 net-scale-factor=0.0039215697906911373
-model-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel
-proto-file=../../../../samples/models/Primary_Detector/resnet10.prototxt
-model-engine-file=../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine
-labelfile-path=../../../../samples/models/Primary_Detector/labels.txt
-int8-calib-file=../../../../samples/models/Primary_Detector/cal_trt.bin
+
+workspace-size=600
+model-file=../../../../samples/models/Primary_Detector_Nano/resnet10.caffemodel
+proto-file=../../../../samples/models/Primary_Detector_Nano/resnet10.prototxt
+labelfile-path=../../../../samples/models/Primary_Detector_Nano/labels.txt
+model-engine-file=../../../../samples/models/Primary_Detector_Nano/resnet10.caffemodel_b3_gpu0_fp16.engine
+
 force-implicit-batch-dim=1
 batch-size=1
 process-mode=1
@@ -74,6 +76,7 @@ num-detected-classes=4
 interval=0
 gie-unique-id=1
 output-blob-names=conv2d_bbox;conv2d_cov/Sigmoid
+cluster-mode=3
 
 [class-attrs-all]
 pre-cluster-threshold=0.2

Any pointer welcome.

2

I think I solved it. I don’t know why this original sample does show that strange X coordinates, but in my derived code I’m working with more than one source. I think I need to divide the derived X coordinate by the number of inputs/sources.

            print("Idx: {0:d}, TS: {1:.8f}, ID: {2:s}, Base: X: {3:d} Y: {4:d}, conf: {5:.8f}".format(
                frame_meta.pad_index,
                (time.time() * 1000000000 - timestamp) / 1000000000, 
                pgie_classes_str[obj_meta.class_id],
                round((obj_meta.rect_params.left + obj_meta.rect_params.width / 2) / self.number_sources), 
                round(obj_meta.rect_params.top + obj_meta.rect_params.height), 
                obj_meta.confidence))
3

Good to know the issue resolved, thanks for the update.

4

BTW: Since I don’t get an answer on this on another thread: Would you know, what the “ntp_timestamp” is exactly?

7

For ntp_timestamp, there are detailed doc NTP Timestamp in DeepStream — DeepStream 6.1.1 Release documentation

I still say it’s better to create a new topic for different issue/question.

8

Thanks! This is what I was looking for. Sorry for not being able to find it by myself.