Dino inference images do not have bounding boxes

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc)
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc)
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here)
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

Are bboxes turned on by default in Dino inference? I just trained an object detection dataset with a map50 evaluation of 31.4. However, when I run inference the output images do not have bboxes. Corresponding label files are created with bboxe coordinates. Is there a spec configuration I am missing?

3

Did you ever refer to its notebook tao_tutorials/notebooks/tao_launcher_starter_kit/dino/dino.ipynb at main · NVIDIA/tao_tutorials · GitHub?

For its reference infer.yaml, tao_tutorials/notebooks/tao_launcher_starter_kit/dino/specs/infer.yaml at main · NVIDIA/tao_tutorials · GitHub, you can find below.

  • conf_threshold: the confidence score threshold
  • color_map: the color mapping for each class. The predicted bbox will be drawn with mapped color for each class

The bboxes are turned on by default in Dino inference. Please check further.

4

Thanks, Morgan. Yes, I have scoured the documentation multiple times. Although I assumed boxes are turned on by default, I didn’t see anything to confirm that. I have set colors for all my classes, but I have not changed the conf_threshold which is set at .50. So you are suggesting that if I lower the threshold this might correct the problem?

5

In followup to your response, I adjusted the conf_threshold to .1 but I am still not getting bboxes. I suspect the issue is a simple solution. I have attached the following spec files to help troubleshoot:

  • train.yaml
  • evaluate.yaml
  • infer.yaml
  • inference.log
    (upload would not allow yaml files so I changed extensions to txt

inference.log (3.5 KB)

train.txt (968 Bytes)
evaluate.txt (476 Bytes)
infer.txt (3.3 KB)
)

6

Could you please open a terminal to debug inside the docker?
Step:

  1. $ docker pull nvcr.io/nvidia/tao/tao-toolkit:5.5.0-pyt /bin/bash
  2. $ docker run --runtime=nvidia -it nvcr.io/nvidia/tao/tao-toolkit:5.5.0-pyt /bin/bash
    Inside the docker, then, run training.
    # dino inference xxx

You can modify /usr/local/lib/python3.10/dist-packages/nvidia_tao_pytorch/cv/dino/model/pl_dino_model.py (See tao_pytorch_backend/nvidia_tao_pytorch/cv/dino/model/pl_dino_model.py at dc07b02eb78c2eb868315107892b466496e55a0f · NVIDIA/tao_pytorch_backend · GitHub and /usr/local/lib/python3.10/dist-packages/nvidia_tao_pytorch/cv/deformable_detr/model/post_process.py (See tao_pytorch_backend/nvidia_tao_pytorch/cv/deformable_detr/model/post_process.py at main · NVIDIA/tao_pytorch_backend · GitHub).

7

Morgan, I ran the dino inference inside the container and it ran without error. I have attached the file, inference.log showing a log of the process. Neither were there bboxes in the images produced from the exercise. I’m assuming the intent was to use any errors from running the inference to make modifications in the pl_dino_model.py or post_process.py files.

Without errors I’m at a loss to make any modifications. I do have a question, though. I noticed the post_process.py file established some METAINFO at the outset of the file which set a tuple of classes and a palette listing tuples of color. Is this a default should the colors not be mapped in the infer.yaml file? Does a color map in the infer.yaml file override this METAINFO? I could see this as a potential hindrance to having bboxes if there is a conflict between the two.

Otherwise, I still don’t see a solution.

Wayne

8

inference.log file attached.

(attachments)

inference.log (3.55 KB)

9

Morgan, I want to update this thread to report that I discovered the issue with the bounding boxes. I didn’t realize that a setting for the number of epochs was limiting the evaluation and inference to only half of the epochs I had trained. Once I corrected that, the bboxes began to appear, though I had to reduce the threshold to .3.

Once I had bboxes I had another issue. All labels were incorrect. Based on the documentation, I had listed my classes in reverse alphabetical order. Once I reverted back to a proper alphabetical order, the labels were all correct.

Thanks for your help, and this topic can be closed.

10

Glad to know it works now. Thanks for the detailed info. Appreciate it.

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.