I cannot reproduce your issue when I download resnet34_peoplenet.tlt(refer to How to run tlt-converter) and run tlt-evaluate against my own dataset.
Refer to the command and spec in PeopleNet v1.0 unpruned model shows very bad results on COCO dataset - #11 by Morganh
The GPU utilization is not 0% during the evaluation.
And it only cost about 1 minute to evaluate 3096 images.
Part of log is as below.
# tlt-evaluate detectnet_v2 -e spec_3class.txt -m resnet34_peoplenet.tlt -k tlt_encode
…
2021-01-23 15:35:01,329 [INFO] iva.detectnet_v2.evaluation.evaluation: step 10 / 194, 0.82s/step
2021-01-23 15:35:04,346 [INFO] iva.detectnet_v2.evaluation.evaluation: step 20 / 194, 0.30s/step
2021-01-23 15:35:07,344 [INFO] iva.detectnet_v2.evaluation.evaluation: step 30 / 194, 0.30s/step
2021-01-23 15:35:10,373 [INFO] iva.detectnet_v2.evaluation.evaluation: step 40 / 194, 0.30s/step
2021-01-23 15:35:13,399 [INFO] iva.detectnet_v2.evaluation.evaluation: step 50 / 194, 0.30s/step
2021-01-23 15:35:16,422 [INFO] iva.detectnet_v2.evaluation.evaluation: step 60 / 194, 0.30s/step
2021-01-23 15:35:19,457 [INFO] iva.detectnet_v2.evaluation.evaluation: step 70 / 194, 0.30s/step
2021-01-23 15:35:22,493 [INFO] iva.detectnet_v2.evaluation.evaluation: step 80 / 194, 0.30s/step
2021-01-23 15:35:25,530 [INFO] iva.detectnet_v2.evaluation.evaluation: step 90 / 194, 0.30s/step
2021-01-23 15:35:28,608 [INFO] iva.detectnet_v2.evaluation.evaluation: step 100 / 194, 0.31s/step
2021-01-23 15:35:31,641 [INFO] iva.detectnet_v2.evaluation.evaluation: step 110 / 194, 0.30s/step
2021-01-23 15:35:34,633 [INFO] iva.detectnet_v2.evaluation.evaluation: step 120 / 194, 0.30s/step
2021-01-23 15:35:37,637 [INFO] iva.detectnet_v2.evaluation.evaluation: step 130 / 194, 0.30s/step
^[[A2021-01-23 15:35:40,690 [INFO] iva.detectnet_v2.evaluation.evaluation: step 140 / 194, 0.31s/step
2021-01-23 15:35:43,677 [INFO] iva.detectnet_v2.evaluation.evaluation: step 150 / 194, 0.30s/step
2021-01-23 15:35:46,685 [INFO] iva.detectnet_v2.evaluation.evaluation: step 160 / 194, 0.30s/step
2021-01-23 15:35:49,712 [INFO] iva.detectnet_v2.evaluation.evaluation: step 170 / 194, 0.30s/step
2021-01-23 15:35:52,709 [INFO] iva.detectnet_v2.evaluation.evaluation: step 180 / 194, 0.30s/step
2021-01-23 15:35:55,771 [INFO] iva.detectnet_v2.evaluation.evaluation: step 190 / 194, 0.31s/step
Matching predictions to ground truth, class 1/3.: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 9872/9872 [00:00<00:00, 16394.45it/s]
Matching predictions to ground truth, class 2/3.: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 7112/7112 [00:00<00:00, 16992.69it/s]
Matching predictions to ground truth, class 3/3.: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 1531/1531 [00:00<00:00, 15919.66it/s]