TAO-Export changes output layers

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hello. I am trying to convert a trained model using the TAO toolkit. My question is when I export the model (whether its fp32 or int8) the output layer before changes from ‘softmax_1’ to ‘argmax_1’ and the output class also changes from (None, 544, 960, 2) to (None, 544, 960, 1)
I need to know what is happening over here and if this is expected behaviour, I need to understand why it is happening. I am attaching the logs and the spec files as well. Please help me regarding this issue and let me know If I need to provide any more information. Thanks in advance!

• Hardware: X86

• Network Type: MASK R-CNN

• TAO Version:
Configuration of the TAO Toolkit Instance
dockers: [‘nvidia/tao/tao-toolkit-tf’, ‘nvidia/tao/tao-toolkit-pyt’, ‘nvidia/tao/tao-toolkit-lm’]
format_version: 2.0
toolkit_version: 3.22.05
published_date: 05/25/2022

• Training spec file
spec_vanilla_unet_rw_finetune.txt (1.7 KB)

  • tao export log:
    export_log.txt (37.4 KB)

  • Pretrained model I am using from Nvidia NGC: peoplesemegnet

2

How did you change? Could you share more detailed step?

3

That is the point. I am not changing the output layer. When I use the below command the output layer gets changed by itself. Forgive me if I don’t understand it as I am new to using TAO toolkit.

!tao unet export --gpu_index=$GPU_INDEX -m $USER_EXPERIMENT_DIR/semseg_experiment_rw_finetune/weights/model_freespace_rw_finetune.tlt \
                -o $USER_EXPERIMENT_DIR/export/model_freespace_rw_finetune.etlt \
                -k $KEY \
                -e $SPECS_DIR/spec_vanilla_unet_rw_finetune.txt \
                --data_type int8 \
                --batches 10 \
                --cal_image_dir $DATA_DOWNLOAD_DIR/real_world_images \
                --engine_file $USER_EXPERIMENT_DIR/export/model_freespace_rw_finetune_int8.engine \
                --cal_cache_file  $USER_EXPERIMENT_DIR/export/model_freespace_rw_finetune_int8_cal.bin \
                --cal_data_file $USER_EXPERIMENT_DIR/export/model_freespace_rw_finetune_int8_data.txt \
                --max_batch_size 4 \
                --batch_size 1

I have a .tlt model using which I am trying to generate .etlt model and the ‘export_log.txt’ is the output logs above command. In that if you in model summary the output layer is ‘softmax_1’ first and then it creates new input and output layers and then it gets changed to ‘argmax_1’.

4

That’s expected behavior.
In the training config, the activation will be used on the last layer. The supported activation is sigmoid or softmax.
If use softmax, the output layer would be replaced with argmax by TAO for optimization. Hence, you need to output-blob-names=argmax_1 in deepstream.

More info can be found in UNET — TAO Toolkit 3.22.05 documentation

BTW, this topic is for Unet. Please modify
Network Type: MASK R-CNN
to
Network Type: Unet

5

Understood. Thanks a lot!

6

@Morganh When attempting to deploy my custom Unet model in DeepStream using the ds-tao-segmentation app (GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream) the included config file states that the output layer of the included unet model is softmax_1. However as stated in this thread the output of user generated models has actually been changed from softmax to argmax during the export. How would you suggest editing either the model export or DeepStream app to accommodate this mismatch in output layers?

7

Please refer to UNET — TAO Toolkit 3.22.05 documentation

## 0=Detector, 1=Classifier, 2=Semantic Segmentation (sigmoid activation), 3=Instance Segmentation, 100=skip nvinfer postprocessing
network-type=100 # set this to 2 if sigmoid activation was used for semantic segmentation

output-tensor-meta=1 # Set this to 1 when network-type is 100
output-blob-names=argmax_1 # If you had used softmax for segmentation model, it would have beedn replaced with argmax by TAO for optimization. Hence, you need to provide argmax_1
8

Yes I’ve read the documentation. The problem I am running into is that all of the recommended prebuilt DeepStream apps that are suggested in these forums for UNET list softmax_1 as the output layer while the TAO docs say softmax was replaced with argmax. The prebuilt apps don’t seem to have been updated to reflect the new output layer of the UNET model.

My question is would it be better to use an older TAO version that does not replace the softmax layer so the model matches the DeepStream apps or try to re-write the apps to accept the new argmax layer?

9

There is no confliction. As mentioned above, in the training config, the activation will be used on the last layer. The supported activation is sigmoid or softmax.
If use softmax, the output layer would be replaced with argmax by TAO for optimization. Hence, you need to output-blob-names=argmax_1 in deepstream.
You can follow the TAO user guide to set the config file. The Deepstream apps config files seems to be not updated.

10

When I run the default example from ds-tao-segmentation I get the following output:

demo
demo1280×720 33.1 KB

The default model is able to classify the cars and people in the input frame.

However when I then swap to using my custom UNET model I don’t get any class variation in the output:

custom
custom1280×720 8.04 KB

I was thinking it was the mismatch of softmax vs argmax and the DeepStream repo not being written to handle the new argmax output layer hence the class detections get lost in the process. Are you saying that the argmax should be processed properly and there may be another reason for the lack of detections?

11

Please create a new topic for your latest comment.
In that new topic, please share the config file, logs, etc. Thanks.

12

Should I create the new post under the TAO or DeepStream forum?

13

Just in TAO forum.

14

Link to new topic for anyone who wants to follow the topic:

Cheers

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.