Overfitting models by using tensorboard

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hi,

I can run the tensorboard in the output directory. But couldn’t see the AP values and validation values. Based on the train loss and validation loss , i can predict the model is over fitting or not … How to see the validation values in the tensorboard

3

Which network did you use for training?

4

Now using Detectnet_v2…

I couldn’t even find training logs when i used retinanet

5

Several comments here.

  1. For detectnet_v2, during training, there should be printing log for AP, mAP or loss. Please double check if you enable below. It will print AP or mAP info every 10 epochs.
validation_period_during_training: 10 
first_validation_epoch: 1

  1. For tensorboard, please refer to Output file when training detectnet 2

  2. For retinanet, there should be training logs. Can you elaborate your issue, why did you say there is no training logs?

6

in Retinanet , i can see the following files in the result directory
retinanet_training_log_resnet.csv weights

So there is no log files. if i run tensorboard on this directory , then i can see the folllowig error in the localhost (tensorboard --logdir . --port 16002)

### No dashboards are active for the current data set.

Probable causes:

* You haven’t written any data to your event files.
* TensorBoard can’t find your event files.

If you’re new to using TensorBoard, and want to find out how to add data and set up your event files, check out the [README](https://github.com/tensorflow/tensorboard/blob/master/README.md) and perhaps the [TensorBoard tutorial](https://www.tensorflow.org/get_started/summaries_and_tensorboard).

If you think TensorBoard is configured properly, please see [the section of the README devoted to missing data problems](https://github.com/tensorflow/tensorboard/blob/master/README.md#my-tensorboard-isnt-showing-any-data-whats-wrong) and consider filing an issue on GitHub.

*Last reload: Tue Jun 09 2020 11:56:03 GMT+0530 (India Standard Time)*

*Data location: .*
7

OK, so there should be no event files. That’s expected. There are event files only in detectnet_v2 network.