• Hardware Platform (Jetson / GPU): Jetson Nano • DeepStream Version: Deepstream 5.0 • JetPack Version (valid for Jetson only): Jetpack 4.4 • TensorRT Version: compiled 7.0 • NVIDIA GPU Driver Version (valid for GPU only)
I kindly would ask about what is the best approach to create a custom model with TLT for age and gender detection?
A classification would not be sufficient as there could be multiple different persons on a frame and also I would like to combine this model with another model.
my questions would be:
-what is the best practice for this topic of gender age, when i would like to use TLT
-would it make sense to use a object detection model and train the face-images, where i would need to generate the kitti data OR would it be necessary to use the facedetect model and (somehow) enhance it with gender/age training
It is necessary to train a classification model firstly. This model aims to differentiate different combination of age and gender. Before training, need to split images into different classes you want to train.
Then use peoplenet model as primary model in Deepstream, and config the classification model as secondary model.
Hello,
thanks for your feedback.
now for this use case, as i have age and gender as a combination, do I end up with one class on the detection or with 2 separate, which gets combined? because i want to get a result: e.g male:0-10.
I mean: for example shall I have the classes (folder structure):
class-A: male:0-10
class-B: female: 0-10
class-C: male: 11-20
…
or is it: class A: male (50% of dataset)
class B: female (remaining 50% of dataset)
class C: 0-10
…which then gets combined.
Hello,
having now created a model for age/gender based on classification and also generated the engine on Jetson, I have used the poeplenet deepstream model and added a secondary-gie0 into the config.
This works for another model which i have trained and loaded to Jetson e.g. detectnet, but when I replace the secondary-gie0 config from detectnet model to the generated classification model and when i start deepstream, the secondary classification model gets loaded but it is not somehow visible (I am running the deepstream with a webcam as input source)
–>What I would expect is, that peoplenet recognizes the face and with the face as input, the classification (age-gender) should recognize the related class and show it as bbox with the class-name.
–>But how to configure this?
below my log from jetson and configs:
log
jetson@nano01:~/tlt-racket/nano-jetson/tlt-configs$ deepstream-app -c my_deepstream_app_source1_peoplenet.txt
2020-09-19 10:14:22.966787: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.2
(deepstream-app:10238): GStreamer-CRITICAL **: 10:14:23.790: passed ‘0’ as denominator for `GstFraction’
(deepstream-app:10238): GStreamer-CRITICAL **: 10:14:23.790: passed ‘0’ as denominator for `GstFraction’
Warning: ‘input-dims’ parameter has been deprecated. Use ‘infer-dims’ instead.
Warning: ‘input-dims’ parameter has been deprecated. Use ‘infer-dims’ instead.
Using winsys: x11
0:00:06.902412681 10238 0x2685430 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 4]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1701> [UID = 4]: deserialized trt engine from :/home/jetson/tlt-racket/nano-jetson/tlt-engines/classification_age_gender.engine
INFO: [Implicit Engine Info]: layers num: 2
0 INPUT kFLOAT input_1 3x224x224
1 OUTPUT kFLOAT predictions/Softmax 37x1x1
0:00:06.902697167 10238 0x2685430 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 4]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1805> [UID = 4]: Use deserialized engine model: /home/jetson/tlt-racket/nano-jetson/tlt-engines/classification_age_gender.engine
0:00:07.090150585 10238 0x2685430 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary_gie_0> [UID 4]: Load new model:/home/jetson/tlt-racket/nano-jetson/tlt-configs/my_config_infer_secondary_age_gender.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
gstnvtracker: Optional NvMOT_RemoveStreams not implemented
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
0:00:07.610666230 10238 0x2685430 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1701> [UID = 1]: deserialized trt engine from :/home/jetson/tlt-racket/nano-jetson/tlt-models/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x544x960
1 OUTPUT kFLOAT output_bbox/BiasAdd 12x34x60
2 OUTPUT kFLOAT output_cov/Sigmoid 3x34x60
0:00:07.610824932 10238 0x2685430 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1805> [UID = 1]: Use deserialized engine model: /home/jetson/tlt-racket/nano-jetson/tlt-models/peoplenet/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine
0:00:07.623889711 10238 0x2685430 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/home/jetson/tlt-racket/nano-jetson/tlt-configs/config_infer_primary_peoplenet.txt sucessfully
Runtime commands:
h: Print this help
q: Quit
p: Pause
r: Resume
NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.
batch-size=1 #Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
gie-unique-id=1
config-file=config_infer_primary_peoplenet.txt
As mentioned in Deepstream on Nano: running detectnet and classification at same time, please confirm you can run /opt/nvidia/deepstream/deepstream/samples/configs/tlt_pretrained_models/deepstream_app_source1_dashcamnet_vehiclemakenet_vehicletypenet.txt successfully.
More, the secondary-gie0 or secondary-gie1 is a classification model
yes, I can run the example.
Also the Peoplenet example works fine. in my created deepstream config, i have added the secondary-gie0 with following settings:
→ this gets successful loaded but without getting the result on the detection.
I think the “opeate-on-class-ids=0” has an effect, because the peoplenet-labels has [Person,Bag,Face] and i would need the face, which i think would be id=2 (if the count starts from 0). I will also create a new model only with 2 classes (gender) and give it a try.
Is there anything i need to check specifically when creating the classification model in TLT?
br, Martin
Sounds like you are working on something fun and exciting.
I have been thinking about similar workflow - 2 step process (1. find an object 2. run text extraction).
When you said that you got it working, I was curious if I understood you correctly.
Did you combine 2 models into 1 by retraining?
Or did you end up using 1 model to find face. Then, another model to figure out the age group?
Hi Jae,
first, i tried with the peoplenet unpruned model and tried to train only with the face-class, but somehow that did not work. Then i went with a separate model running for the age/gender detection, but finally now, I went with the Azure API for the gender/age detection…
Br Martin