Options to retrain Fpenet and which one to use

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

I need some direction of where to start and what is needed. The only thing I would like to do is to re-train the fpenet model with more details.

I don’t have hardware with an NVidia GPU (besides the jetson TX2 NX ) so I thought I’d use a EC2 on AWS .

We are currently using deployable_v2 model from Facial Landmarks Estimation | NVIDIA NGC on deepstream 6.0.1 . I believe this is limited to 80 facial landmarks.
Q1: is 80 correct ?

Going through multiple bits of documentation I believe there are a few ways to run the TAO toolkit to retrain.

  1. on bare metal (not available to us because I don’t have the hardware here)

  2. using AWS EC2 and following the instructions

  3. Running it directly from a docker running ‘docker run -v /root:/root -it nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5 bash’ from here

  4. Apparently, you can also install the TAO toolkit using ‘pip3 install nvidia-tao’ ?

Q2: Are there really 4 different ways ?
Q3: Which way would be the easiest way to install the tao toolkit with the idea of using it to create a new model trained on more images ?

Then after installing the tao-toolkit I need to call the retrain command. From what i see there are 2 options ?

  1. run the fpenet.ipnbk notebook and follow the very confusing instructions regarding all the directories
  2. in the nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5 docker there is a ‘fpenet train’ command
  3. is there a different way to do the same using the nvidia-tao python module ?

Q4: Which one should I use ?

Q5: and are there any examples of the file formats?

Q6: do I need to resize the new input images into a particular size or can i just update the size in the configuration ?

Q7: Is Facial Landmarks Estimation — Transfer Learning Toolkit 3.0 documentation applicable to me (using deepstream 6.0.1 and 80 points, and using tao-toolkit instead of TLT ?) Or does this apply to another system ?

2

Q1: Yes for this model.
Q2: User can run TAO training by using local dgpu machine or Cloud machine.
For local dgpu machine, user can install tao launcher via pip3 install nvidia-tao. Actually it will trigger the corresponding docker. For FPENet, it will pull nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5.
Q3: You can use “docker run” or tao launcher. Actually they are the same because it will run nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5.
Q4: Refer to https://github.com/NVIDIA/tao_tutorials/blob/main/notebooks/tao_launcher_starter_kit/fpenet/fpenet.ipynb. The same as Q2, pip3 install nvidia-tao will install the tao launcher. Then user can run in terminal with command like tao model fpenet xxx. If user run docker run xxx, then user can run command fpenet train xxx inside the docker.
Q5: The valid image extensions are .jpg , .jpeg , and .png .
Q6: Input: Images of (80, 80, 1), refer to Facial Landmarks Estimation - NVIDIA Docs
Q7: Please refer to latest user guide: Facial Landmarks Estimation - NVIDIA Docs
Also, to get started, please use latest 5.0 notebook: https://github.com/NVIDIA/tao_tutorials/blob/main/notebooks/tao_launcher_starter_kit/fpenet/fpenet.ipynb
Source code is also available: https://github.com/NVIDIA/tao_tensorflow1_backend/tree/c7a3926ddddf3911842e057620bceb45bb5303cc/nvidia_tao_tf1/cv/fpenet.

3

@Morganh thank you for that. I’ve gone with the pip3 install nvidia-tao option.

regarding Q6: Input images of (80,80,1) does that mean the images that you supply to the tao fpenet train .... need to be 80x80 pixels in grayscale ? would it need to already be cropped to just the face ?

4

pip3 install nvidia-tao
seems to start up tao-toolkit:4.0.0-tf1.15.5 ?

$ tao fpenet dataset_convert -e /workspace/experiments/dataset_config.txt
2023-10-04 04:31:56,871 [INFO] root: Registry: ['nvcr.io']
2023-10-04 04:31:56,938 [INFO] tlt.components.instance_handler.local_instance: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:4.0.0-tf1.15.5

i installed conda and setup the environment like:
conda create -n launcher python=3.6
because that is what is in the TAO Toolkit Quick Start Guide - NVIDIA Docs

is that the reason why it is using the old tao-toolkit ?

5

Yes,

It happened to me, if you install 3.7 or higher it will download and use the 5.0 version.

1 Like
7

thank you !!

8