YOLO v4 Tao runnig evaluation on a NON validation dataset

[chijco]

Hi I have a model which is already trained, pruned and retrained
and now I have a totally new ground truth labeled dataset that I want to check my model with it.
option A- If I run inference I do not get there an option to get mAP (I think)
option 2 - I can run evaluation BUT, here you say that evaluation must be run on THE SAME training-specification file

Thus I can not change there the paths using new validation directrory. So what should I do?
Thanks in advance

[Morganh]

You can use option2. To run evaluation.
You can use the new path to the new validation directory.

[chijco]

Oh I already tried…
When I get to the cell of the retrained model, I did like this:

Here is a screen from both specs files:

Please help me solve this issue. Thank you

[Morganh]

Can you share below?

• full log
• One or two new label files

[chijco]

here is s label example:
18-26-41_84_RGB.txt (1020 Bytes)

Here are 2 logs:
1st one is when the evaluation DOES work - when I run it using the original specs file:

log_where_evaluation_DOES_work (2).txt (2.4 MB)

2nd log is where the evaluation DID NOTwork:
log_where_evaluation_does NOT_work.txt (128.9 KB)

Also here you can see how I build my docker paths in the notebook cell:

[Morganh]

Please change the label files.

Got label marked as difficult(occlusion > 0), please set occlusion field in KITTI label to 0

Refer to Data Annotation Format - NVIDIA Docs

For example, please change to

someclass1 0 0 0 317.2704700000002 182.39025750000013 360.75424000000027 232.31606750000017 0 0 0 0 0 0 0

[chijco]

Wow! First of all thank you, it did help!
Let be ask you something please.
Actually if you noticed I had 4 datasets each included training and validation folders. What I did was taking one of them putting in some new directory and tried to run evaluation on it like if it was new.
Now When I look at my 4 datasets 3 of them have this strange -1 value in occlusion instead of 0. Do you think it can also be a reason to relatively poor training mAP (about 0.4-0.5)?

[Morganh]

You can set occlusion to 0 in all the labels. And train again.

[chijco]

I am on it, thank you so much again!
Please answer me in another place about if it is posible to make automatc split to train and val without tfrecords

[Morganh]

@chijco

Please answer me in another place about if it is posible to make automatc split to train and val without tfrecords

Sync in Splitting dataset without tfrecords in TAO YOLOV4 and Setting multiple directories for training and validation in yolo_v4_train_resnet18.txt