People Net -

For your case, actually it is a new training. The peoplenet model contains pretrained weights that may be used as a better starting point for people class.
I also do similar experiment on my side. I train the “People” class and a new class “cart”.
Prepare some data for both classes. All the data are 960x544.
Then set the training spec and also tune the class_weight.
The unpruned peoplenet pretrianed model works as a good pretrained weight.

Your new model with the people and carts is as good as the original peoplenet model when it comes to detecting people?

Also, thank you for doing this experiment.

Actually it is a new training because a new class is added.
I prepare 14k person data and 3.7k cart data, run totaly 10 epochs, 40 minutes. The AP for Person is about 60%.
I did not finetune any hyper-parameters a lot.
So, after finetune or run longer, I believe the mAP can still improve further.

Would it be possible to share your person dataset?

Sorry, this data is from Nvidia internal only.

I understand, thanks.

Good Morning,

In relation to transfer learning, would it be possible to leverage the already good PeopleNet to be able to detect gray images (Infrared Cameras).

Retraining completely is not possible as "PeopleNet v1.0 model was trained on a proprietary dataset with more than 5 million objects for person class. "

If we can use transfer learning to retain the networks ability to detect people in colour images and gray scale images with the same level of accuracy

If so how many new “gray” images would we need to use? How high is the risk of over training the network with the new gray images

1 Like

See https://ngc.nvidia.com/catalog/models/nvidia:tlt_peoplenet

Dark-lighting, Monochrome or Infrared Camera Images

The PeopleNet model was trained on RGB images in good lighting conditions. Therefore, images captured in dark lighting conditions or a monochrome image or IR camera image may not provide good detection results.

More reference:

For training on gray scale images only, please consider to set

output_image_channel: 1

About how many images need to use, refer to Dataset Practices

Thanks

I was aware Peoplenet was not trained for gray scale images so I want to be able to detect people in the day time and at night.

As for retraining I thought the purpose of transfer learning was to reduce the need for huge amounts of new data ?

The link you gave just tells the amount of data used to train the network for RBG images at different distances, half indoors half outdoors?

Does this mean we would need a similar number of images from IR cameras, and will this not reduce the detections of colour images since we cant add Nvidias Training Images to the dataset

@Andrew_Smith
No, you need not a similar number of images. That is why unpruned peoplenet model is provided in ngc. User can set it as pretrained model and train their own data. If your data are colour images, the transfer learning should run smoothly. But as the link said, “monochrome image or IR camera image may not provide good detection results”, that is the known limitation.

Thank you for the response.

I need to detect both daytime and night time camera images.

So should I disregard PeopleNet?

Does this mean the Unpruned PeopleNet will not be able to be trained to recognize IR camera images?

If I aquire a large sum of IR camera images, will training on the unpruned model completely ruin the Colour Image detection afterwards

Actually several months ago, I download a public IR dataset and train them with TLT detectnet_v2. The mAP can reach about 80%. So we can not say the peoplenet cannot work for IR images. Just say it May have a lower mAP because the peoplenet is trained with colour images.
For your case, the unpruned peoplenet is sure to train and recognize IR images. You can prepare the data and train. Set the unpruned peoplenet model as pretrained model. It is a good start point to detect people/bag/face.

Okay Thank you so much for clarifying that for me

Its great to hear as I am excited to use the Jetson range.

Would you be able to share a link to the the Dataset? If by public you mean in the public domain

FLIR dataset https://www.flir.asia/oem/adas/adas-dataset-form/

@Morganh Are you aware of any dataset resources in the public domain that contain normal RGB images of people annotations ?

You can search it. I did not search previously because it is enough in NV internal.

Got it, thanks.

@Morganh can you share the environment setup of running peoplenet as i amnot able to build the docker so i can setup the environment individually

@abhigoku10
Please create a new topic in TLT forum.
For TLT 3.0 environment, please refer to Requirements and Installation — Transfer Learning Toolkit 3.0 documentation and TLT Launcher — Transfer Learning Toolkit 3.0 documentation