Retrained Peoplenet Model Not Detecting

Hi,

We earlier retrained peoplenet (detecnet with resnet34) to detect age and gender. The classes of our dataset were female_adult, male_adult, and child.

When we deploy the model with the deepstream occupancy analytics application, on jetson nano with deepstream 5.1, the model doesn’t show any bounding boxes around people appearing in the frame, and also doesn’t count any of them.

The app runs without errors, but we see no results. Could you help us try to figure out how to go about solving this issue?

Thanks

After training, did you run “tao detectnet_v2 inference xxx” to check whether the people can be detected?

Hi @Morganh

Yes we did that. Below is the result we got:

The pipeline seems to be wrong. The peoplenet is an object-detection model. It cannot classify age and gender.
So, for your case, it is needed to run pgie(peoplenet) and then sgie(classification model) .

Hi @Morganh

We changed the pipeline to include Peoplenet as PGIE and the retrained version of it to detect age and gender as SGIE.
What we noticed is that we get a bounding box around the person but the label is always person with no indication of the age and gender labels which are (female_adult, and male_adult).

The picture below is and example of the inference we get:

Please find below the PGIE and SGIE configuration files as well as the test5 deepstream run file:
config_infer_primary_peoplenet.txt (2.2 KB)
test5_config_file_src_infer_tlt.txt (5.0 KB)

How did you train a model to classify age and gender ? Please make sure you are using classification network.

We basically took images of men and women and children and labeled them: female_adult, male_adult, and child instead of just Person.
Can you please specify what you mean by using a classification network in the TAO context?

Thanks

For this kind of task, we can use a classification model instead of an object-detection model.
Similar example, refer to deepstream application.


samples/configs/tlt_pretrained_models/deepstream_app_source1_dashcamnet_vehiclemakenet_vehicletypenet.txt

Hey @Morganh

Do you mean we need to retrain detectnet again but as a classification model?

I modified the files I included before as follows, can you please take a look:
config_infer_primary_peoplenet.txt (2.2 KB)
config_infer_secondary_age_gender.txt (2.8 KB)
test5_config_file_src_infer_tlt.txt (4.9 KB)

When I run the app now I get this message:

(deepstream-test5-analytics:11038): GLib-CRITICAL **: 11:25:46.641: g_strchug: assertion 'string != NULL' failed

(deepstream-test5-analytics:11038): GLib-CRITICAL **: 11:25:46.641: g_strchomp: assertion 'string != NULL' failed
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.
Unknown or legacy key specified 'input-tensor-from-meta' for group [property]
Warning: 'input-dims' parameter has been deprecated. Use 'infer-dims' instead.

Using winsys: x11 

0:00:25.419270864 11038   0x559c3e6700 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1702> [UID = 2]: deserialized trt engine from :/opt/nvidia/deepstream/deepstream-5.1/sources/apps/sample_apps/deepstream-occupancy-analytics-AG-1/config/Age_and_Gender/experiment_dir_final/resnet34_detector.trt
INFO: [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT input_1         3x384x1248      
1   OUTPUT kFLOAT output_bbox/BiasAdd 12x24x78        
2   OUTPUT kFLOAT output_cov/Sigmoid 3x24x78         

0:00:25.420019010 11038   0x559c3e6700 WARN                 nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1659> [UID = 2]: backend can not support dims:3x544x960
0:00:25.420056199 11038   0x559c3e6700 WARN                 nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1814> [UID = 2]: deserialized backend context :/opt/nvidia/deepstream/deepstream-5.1/sources/apps/sample_apps/deepstream-occupancy-analytics-AG-1/config/Age_and_Gender/experiment_dir_final/resnet34_detector.trt failed to match config params, trying rebuild
0:00:26.703944611 11038   0x559c3e6700 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1716> [UID = 2]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
ERROR: [TRT]: UffParser: Output error: Output predictions/Softmax not found
parseModel: Failed to parse UFF model
ERROR: failed to build network since parsing model errors.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:28.974369420 11038   0x559c3e6700 ERROR                nvinfer gstnvinfer.cpp:613:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1736> [UID = 2]: build engine file failed
Segmentation fault (core dumped)

No.
For the pipeline, as mentioned above, please

  1. use TAO detectent_v2 network to train an object detection model. Or you can directly use peoplenet model. This model can detect person. This model is pgie.
  2. Then, use TAO classification network to train a classification model. See more in Image Classification — TAO Toolkit 3.22.05 documentation. For the training dataset, in your case, there should be three folders. The 1st folder contains the cropped images of female_adult. The 2nd folder contains the cropped images of male_adult. The 3rd folder contains the cropped images of child. After training done, this model can set to sgie.

Hi @Morganh

Could you please specify what you meant by cropped images. We will be retraining Resnet34 using the classification network, and the data we have is images: 1224x 370, that were labeled as kitty format. Do we need to change any of that?
Please find below an example of data from the dataset:

410.txt (42 Bytes)

Hi,
The cropped images are used to train a classification network. When user train a classification network, the kitti format labels are not needed. I suggest you to download the classification jupyter notebook(TAO Toolkit Quick Start Guide — TAO Toolkit 3.22.05 documentation) and have a try to get familiar the process.
In the 1st folder contains the cropped images of female_adult, I mean that it is better for each image to contain female_adult instead of female_adult+male_adult. I assume some training images of your original dataset contain all kinds of classes(female_adult, male_adult and child), so that is the reason I suggest you to crop it.

Hi @Morganh

I am following your advice here to run the classification network. The issue I found is that when I run the classification network from our VM on GCP, in the first cell where it creates the local directories such as the folder tao-experiments, data, etc…
I can’t seem to find them on the home directory as expected.

We used the VLM before to run detectnet and this step worked fine and still does even now, so we renamed the old tao-experiments folder, and still, the classification network doesn’t create the folder correctly.

I tried creating a new environment and downloaded the dependencies, and the samples again to launch the classification network from there, but I still get the same issue.

Do you have an idea how we can fix this?

Is it related to issue mentioned in Chmod: cannot access '/opt/ngccli/ngc': No such file or directory - #2 by Morganh

No actually, it’s the first cell of the jupyter notebook, where we set up the local directories on the VM where we will later download the data and the different versions of the trained model. The directory looks like this:
tao-experiments—data
—classification

In our case, we get a message saying that the directory has been created successfully but we can’t find any on /home. I tried creating everything manually but at the cell where we load the data to the notebook for the training, we get an error message saying that there is no directory called tao-experiments which is confusing.

Please create a new topic and share the full log as well. Thanks.

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.