Hi all,
Description
I want to train detectnet-v2 + res 18 on kitti dataset, I mounted the docker container on host system and load the kitti dataset as offline and then convert to tfrecord and then train it only one epoch. and I followed from TLT document.
Matching predictions to ground truth, class 1/3
Epoch 1/1
=========================
Validation cost: 0.000191
Mean average_precision (in %): 0.0006
class name average precision (in %)
------------ --------------------------
car 0.00165432
cyclist 0
pedestrian 0
Median Inference Time: 0.015159
INFO:tensorflow:Saving checkpoints for step-562.
2020-05-29 09:06:21,303 [INFO] tensorflow: Saving checkpoints for step-562.
Time taken to run iva.detectnet_v2.scripts.train:main: 0:06:15.723442.
Question:
1 - Loss start from 0.1 and fall down to 0.001, It’s strange for one epoch, but I don’t know it’s correct or no?
2- when I use restnet18.hdf5 for retrained, then this weight is for whole network or feature extraction only?
3- validation loss is very small, but average precision of all class is zeros, why?
Environment
TensorRT Version : 7.1.0 [Developer Preview]
GPU Type : Jetson Nano
Nvidia Driver Version : JetPack-4.4 DP (L4T R32.4.2)
CUDA Version : 10.2
CUDNN Version : 8.0.0 [Develop Preview]
Operating System + Version : Ubuntu 18.04, Linux kernel 4.9.140
Python Version (if applicable) : 3.6.9
Baremetal or Container (if container which image + tag) : Baremetal