I think no. See below result.
$ cat 235681_result.txt |grep " \[79\]" |wc -l
5450
It is similar to the total frames of the test video.
Thank you. I’ve restarted my machine over the weekend and i now also only get 1 layer per frame… i must have done something wrong previously. Sorry about that.
I extracted frame 239
and ran the app on it:
jeteye@jeteye:/data/src/taoapps/deepstream_tao_apps/apps/tao_others/deepstream-faciallandmark-app$ ./testframe.sh
+ export LD_LIBRARY_PATH=:/opt/nvidia/deepstream/deepstream/lib/cvcore_libs
+ pwd
+ currentdir=/data/src/taoapps/deepstream_tao_apps/apps/tao_others/deepstream-faciallandmark-app
+ ./deepstream-faciallandmark-app 2 ../../../configs/facial_tao/sample_faciallandmarks_config.txt file:///data/src/taoapps/deepstream_tao_apps/apps/tao_others/deepstream-faciallandmark-app/frame239_1.jpg ./landmarks
Request sink_0 pad from streammux
Now playing: file:///data/src/taoapps/deepstream_tao_apps/apps/tao_others/deepstream-faciallandmark-app/frame239_1.jpg
0:00:04.318385291 26995 0x55769a32d0 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<second-infer-engine1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1900> [UID = 2]: deserialized trt engine from :/data/src/taoapps/deepstream_tao_apps/models/faciallandmark/faciallandmarks.etlt_b32_gpu0_fp16.engine
INFO: [FullDims Engine Info]: layers num: 4
0 INPUT kFLOAT input_face_images 1x80x80 min: 1x1x80x80 opt: 32x1x80x80 Max: 32x1x80x80
1 OUTPUT kFLOAT conv_keypoints_m80 80x80x80 min: 0 opt: 0 Max: 0
2 OUTPUT kFLOAT softargmax 80x2 min: 0 opt: 0 Max: 0
3 OUTPUT kFLOAT softargmax:1 80 min: 0 opt: 0 Max: 0
ERROR: [TRT]: 3: Cannot find binding of given name: softargmax,softargmax:1,conv_keypoints_m80
0:00:04.319682312 26995 0x55769a32d0 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<second-infer-engine1> NvDsInferContext[UID 2]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1868> [UID = 2]: Could not find output layer 'softargmax,softargmax:1,conv_keypoints_m80' in engine
0:00:04.319712970 26995 0x55769a32d0 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<second-infer-engine1> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2004> [UID = 2]: Use deserialized engine model: /data/src/taoapps/deepstream_tao_apps/models/faciallandmark/faciallandmarks.etlt_b32_gpu0_fp16.engine
0:00:05.028879020 26995 0x55769a32d0 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<second-infer-engine1> [UID 2]: Load new model:../../../configs/facial_tao/faciallandmark_sgie_config.txt sucessfully
0:00:05.029167674 26995 0x55769a32d0 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1161> [UID = 1]: Warning, OpenCV has been deprecated. Using NMS for clustering instead of cv::groupRectangles with topK = 20 and NMS Threshold = 0.5
0:00:05.490353353 26995 0x55769a32d0 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1900> [UID = 1]: deserialized trt engine from :/data/src/taoapps/deepstream_tao_apps/models/faciallandmark/facenet.etlt_b1_gpu0_fp16.engine
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x416x736
1 OUTPUT kFLOAT output_bbox/BiasAdd 4x26x46
2 OUTPUT kFLOAT output_cov/Sigmoid 1x26x46
0:00:05.491759276 26995 0x55769a32d0 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2004> [UID = 1]: Use deserialized engine model: /data/src/taoapps/deepstream_tao_apps/models/faciallandmark/facenet.etlt_b1_gpu0_fp16.engine
0:00:05.553291282 26995 0x55769a32d0 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-infer-engine1> [UID 1]: Load new model:../../../configs/facial_tao/config_infer_primary_facenet.txt sucessfully
Decodebin child added: source
Decodebin child added: decodebin0
Running...
Decodebin child added: nvjpegdec0
In cb_newpad
###Decodebin pick nvidia decoder plugin.
secondary buffer probe frame number: 0
meta type: 12
frame number 0 received points
frame number 0 layername: softargmax:1
score: [0] score: 0.159058
score: [1] score: 0.138916
score: [2] score: 0.162354
score: [3] score: 0.117737
score: [4] score: 0.0539246
score: [5] score: 0.0953369
score: [6] score: 0.123779
score: [7] score: 0.352051
score: [8] score: 0.393799
score: [9] score: 0.189453
score: [10] score: 0.102295
score: [11] score: 0.0842896
score: [12] score: 0.230225
score: [13] score: 0.21582
score: [14] score: 0.153564
score: [15] score: 0.103455
score: [16] score: 0.1203
score: [17] score: 0.214722
score: [18] score: 0.190918
score: [19] score: 0.158447
score: [20] score: 0.169189
score: [21] score: 0.348633
score: [22] score: 0.398682
score: [23] score: 0.451416
score: [24] score: 0.470703
score: [25] score: 0.388672
score: [26] score: 0.261719
score: [27] score: 0.270996
score: [28] score: 0.187866
score: [29] score: 0.196899
score: [30] score: 0.25415
score: [31] score: 0.435303
score: [32] score: 0.397217
score: [33] score: 0.180054
score: [34] score: 0.21582
score: [35] score: 0.164795
score: [36] score: 0.365479
score: [37] score: 0.305176
score: [38] score: 0.291504
score: [39] score: 0.225098
score: [40] score: 0.231201
score: [41] score: 0.319336
score: [42] score: 0.0450134
score: [43] score: 0.0773315
score: [44] score: 0.0809326
score: [45] score: 0.136597
score: [46] score: 0.054657
score: [47] score: 0.0906372
score: [48] score: 0.322266
score: [49] score: 0.237671
score: [50] score: 0.311035
score: [51] score: 0.400635
score: [52] score: 0.239746
score: [53] score: 0.24353
score: [54] score: 0.213501
score: [55] score: 0.274902
score: [56] score: 0.391357
score: [57] score: 0.394043
score: [58] score: 0.386963
score: [59] score: 0.276123
score: [60] score: 0.314697
score: [61] score: 0.266846
score: [62] score: 0.27002
score: [63] score: 0.456299
score: [64] score: 0.265381
score: [65] score: 0.49707
score: [66] score: 0.376465
score: [67] score: 0.412598
score: [68] score: 0.383545
score: [69] score: 0.377686
score: [70] score: 0.356445
score: [71] score: 0.306396
score: [72] score: 0.06604
score: [73] score: 0.0854492
score: [74] score: 0.115417
score: [75] score: 0.0625
score: [76] score: 0.325195
score: [77] score: 0.896484
score: [78] score: 0.0953979
score: [79] score: 0.119934
End of stream
Returned, stopping playback
Average fps 0.000233
Totally 1 faces are inferred
Deleting pipeline
so there is one point with a high accuracy:
score: [77] score: 0.896484
and the rest is low-ish.
77 is 0 based, so for the scorecard https://developer.nvidia.com/sites/default/files/akamai/TLT/fpe_sample_keypoints.png it is 78 which is the left ear.
Hi,
For the lower value/range of confidence, I think it is related to the training dataset and test dataset.
Please go back and help run some experiments with the default jupyter notebook.
Step:
- Run training with the afw dataset mentioned in the default notebook.
- After training is done, get the model.tlt.
- And then config this model.tlt into deepstream, and run inference against the afw test video/image you have prepared last week.
- Check the confidence value/range.
Thanks a lot.
Please hang on while we retrain against the AFW dataset. it will take a bit of time.
Is it known what the average confidence is of the model against the dataset that the model was trained on.
I think if i heard correctly the model was not trained against the AFW dataset ?
When i run it against that i get:
Returned, stopping playback
Average fps 13.881726
number of facial landmarks: 114640
total confidence 29045.9
average 0.253366
Totally 1439 faces are inferred
so an average confidence of 0.25 or 25%
If i run it on our own dataset it looks similar ( 26%)
Is it known what the confidence is against the dataset that the nvidia deepstream example model was trained on ? The model card only states the pixel distance, not the output of the confidence from the inference.
Yes, refer to (Facial Landmarks Estimation | NVIDIA NGC), the model in ngc was trained on a combination of NVIDIA internal dataset and Multi-PIE dataset. NVIDIA internal data has approximately 500k images and Multipie has 750k images.
No, we do not know. We did not collect the result of confidence value.
We retrained against our own dataset and the confidence went up to 0.75
thank you for following along, i think i’ve got a better understanding of what to expect from the supplied models now.
maybe the model card of the TAO models should have an indication of the to-be expected confidence and that when re-training this confidence will increase a lot quickly ?
again thank you for your help !
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.