Yolov4, no bounding box was showed when using Re-Train Model

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc) Jetson nano
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc) Yolov4
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here) v3.22.05-tf1.15.4-py3
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

Hi,
I use Tao yolov4 to create a custom detector.

  • After training for the first time, I got the unpruned model, which is pretty good (83% of the map). Then I transfer the model to .etlt and run it with deepstream (Integrate the .etlt model directly in the DeepStream app) → the result runs fine

  • Next I prune model, prune ratio is 0.8

  • Finally retrain with prune model. After 500 epochs, the map is 80%. I deploy it with deepstream (using both tao-converter and DeepStream app) but the result is very bad, no object detected.

Please help, I don’t know where I am going wrong.

2

More information:
Based on the log when retraining, I see the map is 0.8. However, I test again with the data “train”, but no bounding box shows up. Is there some problem?

Command

yolo_v4 inference -i data/images/ -o data/output -e specs/yolo_v4_retrain_resnet18_kitti.txt -m experiment_dir_retrain/weights/yolov4_resnet18_epoch_350.tlt -k threat_detection -t 0.01
3

This retrained pruned model runs ok on deepstream 6.2 RTX. but under jetson nano deepstream vesion 6.0.1 still error

4

@mchi Can you pls help me to check it?

5

do you mean the accuracy is not good or fail to build the TRT engine and run the application?

if it failed on build the TRT engine and run the application, please share the error log.

6

Hi @mchi Thank for your reply

I use TAO-nvidia yolov4 to train detector. Therefore, when training is complete, I get 2 models including unpruned model and retrain pruned model.
For Unprune model (map 0.84):

  • I run fine on both RTX and jetson nano (fp16)

For retrain pruned model (map 0.8):

  • I run fine on RTX deepstream 6.2 (fp16)
  • On jetson nano (fp16): I tried using tao-converter and deepstream app to generate trt engine file. All are error free, but do not show bounding box.
7

So on the same jetson nano. Same code, same config, same post processor, same way of generating .etlt files. But the unpruned model detects the object and the “retrain pruned model” detects nothing.

Try again with RTX card, retrain pruned model can detect object.

8

log_create_trt_engine_using_deepstream_app.txt (6.1 KB)
log_create_trt_engine_using_tao_converter.txt (13.3 KB)
If you need the config or model file, just tell me, I can send it to you privately

10

Firstly, to narrow down, when you finish retraining the pruned model, please run “yolo_v4 evaluation” and “yolo_v4 inference”. For “yolo_v4 inference”, please try different threshold.

11

Hi @Morganh Thank for your help. I fixed the bug above. But I still don’t know if the fault is in DS or TAO. (May be TAO, i guess).
Buti still do not understand why the same tlt model, i can run on RTX, cannot run on Jetson nano.
I check yolov4 inference, It seems that our yolov4 model works, it can detects our interested objects.
I check with Deepstream on RTX, it works perfect.

12

Thanks for the info. Glad to know the issue is gone now. May I know that which is changed for fix?

13

Hi @Morganh

when prune model, i just replace augment eq intersection to union. Then retrain

Today is the weekend that you still work + TAO/DS are incredible. As a computer vision dev, we are grateful to you and your team for helping us.

14

Great. Thanks a lot for the info.

1 Like
15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.