Incorrect "arch" value for efficientnet_b1_relu from the NGC pretrained model repo

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

T4 GPU
deepstream:6.0-triton
TensorRT Version=8.0.1
NVIDIA GPU Driver Version 470.82.01

After pulling down the efficientnet_b1_relu model from the NGC (shown below) the “arch” value needed to allow its use does not seem to match. The value that “seems” to work is efficientnet_b0. Is this correct? (I tried several others that did not work :
efficientnet_b1_relu ------- “unsupported backbone”
efficientnet_b1_swish ---- “unsupported backbone”
efficientnet_relu ------------ “unsupported backbone”
efficientnet_b1 ------------ “unsupported backbone”
).

Using efficientnet_b0 the model then builds but has 11.1 million parameters, efficientnet_b1 is listed in some documents as having 7.8 million and efficientnet_b3 as having about 12 million so also wondering if the model being downloaded below is actually b1 … any help in figuring out both the actual model and the correct “arch” label is greatly appreciated! Thanks!

Pull pretrained model from NGC

!ngc registry model download-version nvidia/tao/pretrained_object_detection:efficientnet_b1_relu
–dest $LOCAL_EXPERIMENT_DIR/pretrained_efficientnet_b1_relu
{
“download_end”: “2022-02-02 12:01:21.778968”,
“download_start”: “2022-02-02 12:01:05.759937”,
“download_time”: “16s”,
“files_downloaded”: 1,
“local_path”: “/home/dell/TAO_CV_experiments_brandt_2/retinanet4/pretrained_efficientnet_b1_relu/pretrained_object_detection_vefficientnet_b1_relu”,
“size_downloaded”: “24.28 MB”,
“status”: “Completed”,
“transfer_id”: “pretrained_object_detection_vefficientnet_b1_relu”
}

3

This is TAO model. Please raise topic in TAO forum. Latest Intelligent Video Analytics/TAO Toolkit topics - NVIDIA Developer Forums

@Morganh

4

Moving to TAO forum.

5

May I know where did you find the info “unsupported backbone” ?
More, which network did you run?
And could you refer to https://docs.nvidia.com/tao/tao-toolkit/text/open_images/open_images_pt_efficientdet.html and TAO Pretrained EfficientDet | NVIDIA NGC ?

6

Hi Morgan, thank you very much for your reply. I’ve just referred to your link regarding the support for Efficientnet within Efficientdet. I should have mentioned I am running the RetinaNet network, and I believe the only two EfficientNets backbones that are shown in the NVIDIA documentation as being compatible for RetinaNet are the efficient_b1_relu and efficient_b1_swish. When I invoke the Tao train on that model, if the config file lists “arch” as anything other than efficientnet_b0, it ends up returning “unsupported backbone” …

7

Thanks for the info, for retinanet, the backbone “Efficientnet B0” can be supported. But retinanet does not support other Efficientnet backbones.
You can refer to https://docs.nvidia.com/tao/tao-toolkit/text/open_images/overview.html#overview

8

Thanks again for helping to figure this out… that would explain why tao was not happy with any other “arch” value. So part of the mystery is solved :-)

However, when I execute the following line from the retinanet.ipynb only efficient_b1_relu and efficient_b1_swish show up as available – wondering why this might be the case ? :

!ngc registry model list nvidia/tao/pretrained_object_detection:*

I get this:

[{
    "accuracyReached": 77.56,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:15:14.015Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "153.7",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 161183816,
    "versionId": "vgg19"
},{
    "accuracyReached": 77.17,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:14:53.744Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "113.2",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 118655144,
    "versionId": "vgg16"
},{
    "accuracyReached": 65.13,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:16:58.199Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "6.5",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 6776712,
    "versionId": "squeezenet"
},{
    "accuracyReached": 77.91,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:12:45.414Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "294.2",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 308488496,
    "versionId": "resnet50"
},{
    "accuracyReached": 77.04,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:12:15.949Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "170.7",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 178944632,
    "versionId": "resnet34"
},{
    "accuracyReached": 76.74,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:12:01.650Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "89.0",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 93278448,
    "versionId": "resnet18"
},{
    "accuracyReached": 77.78,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:13:35.052Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "576.3",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 604328880,
    "versionId": "resnet101"
},{
    "accuracyReached": 74.38,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:11:51.295Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "38.3",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 40173128,
    "versionId": "resnet10"
},{
    "accuracyReached": 72.75,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:16:51.593Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "5.0",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 5258048,
    "versionId": "mobilenet_v2"
},{
    "accuracyReached": 79.5,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:16:42.810Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "26.2",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 27489360,
    "versionId": "mobilenet_v1"
},{
    "accuracyReached": 77.11,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:17:04.776Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "47.6",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 49958952,
    "versionId": "googlenet"
},{
    "accuracyReached": 77.11,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:12:16.058Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "26.8",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 28082680,
    "versionId": "efficientnet_b1_swish"
},{
    "accuracyReached": 77.11,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:11:58.715Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "26.8",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 28082608,
    "versionId": "efficientnet_b1_relu"
},{
    "accuracyReached": 76.44,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:15:38.311Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "311.7",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 326824240,
    "versionId": "darknet53"
},{
    "accuracyReached": 77.52,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:16:17.364Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "152.8",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 160242408,
    "versionId": "darknet19"
},{
    "accuracyReached": 77.1,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-11-23T07:40:58.799Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "28.6",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 29955696,
    "versionId": "cspdarknet_tiny"
},{
    "accuracyReached": 76.44,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-09-10T00:55:18.313Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "103.0",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 107993624,
    "versionId": "cspdarknet53"
},{
    "accuracyReached": 77.52,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-09-10T00:55:09.000Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "62.9",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 65913984,
    "versionId": "cspdarknet19"
}]
9

using that list the notebook then uses:

# Pull pretrained model from NGC
!ngc registry model download-version nvidia/tao/pretrained_object_detection:efficientnet_b1_relu \
                    --dest $LOCAL_EXPERIMENT_DIR/pretrained_efficientnet_b1_relu
10

Sorry for the confusion. For pretrained models of efficientnet b0~b5, you can refer to https://docs.nvidia.com/tao/tao-toolkit/text/open_images/open_images_pt_efficientdet.html
→ TAO Pretrained EfficientDet | NVIDIA NGC and

ngc registry model list nvidia/tao/pretrained_efficientdet:*
11

that sounds good, would those all be compatible with the retinanet object detection? under the object detection directory only those b1_relu and b1_swish show up and the b1_relu (combined with the retinanet) then shows about 11.1 million parameters so am thinking it might really be a b1, though accessed with the arch name efficientnet_b0 … idk

12

As mentioned above, for retinanet, the backbone “Efficientnet B0” can be supported. B1~B5 are not supported.

13

ok that’s good to know and matches the current guide for RetinaNet

arch     Backbone for feature extraction. Currently, “resnet”, “vgg”, “darknet”, “googlenet”, “mobilenet_v1”, “mobilenet_v2” and “squeezenet”, “efficientnet_b0” are supported.

However the line given by the NVIDIA notebook for object detection backbones

!ngc registry model list nvidia/tao/pretrained_object_detection:*

ends up listing the following (implying support for object detection):

[{
    "accuracyReached": 77.56,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:15:14.015Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "153.7",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 161183816,
    "versionId": "vgg19"
},{
    "accuracyReached": 77.17,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:14:53.744Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "113.2",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 118655144,
    "versionId": "vgg16"
},{
    "accuracyReached": 65.13,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:16:58.199Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "6.5",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 6776712,
    "versionId": "squeezenet"
},{
    "accuracyReached": 77.91,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:12:45.414Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "294.2",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 308488496,
    "versionId": "resnet50"
},{
    "accuracyReached": 77.04,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:12:15.949Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "170.7",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 178944632,
    "versionId": "resnet34"
},{
    "accuracyReached": 76.74,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:12:01.650Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "89.0",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 93278448,
    "versionId": "resnet18"
},{
    "accuracyReached": 77.78,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:13:35.052Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "576.3",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 604328880,
    "versionId": "resnet101"
},{
    "accuracyReached": 74.38,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:11:51.295Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "38.3",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 40173128,
    "versionId": "resnet10"
},{
    "accuracyReached": 72.75,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:16:51.593Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "5.0",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 5258048,
    "versionId": "mobilenet_v2"
},{
    "accuracyReached": 79.5,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:16:42.810Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "26.2",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 27489360,
    "versionId": "mobilenet_v1"
},{
    "accuracyReached": 77.11,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:17:04.776Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "47.6",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 49958952,
    "versionId": "googlenet"
},{
    "accuracyReached": 77.11,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:12:16.058Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "26.8",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 28082680,
    "versionId": "efficientnet_b1_swish"
},{
    "accuracyReached": 77.11,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:11:58.715Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "26.8",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 28082608,
    "versionId": "efficientnet_b1_relu"
},{
    "accuracyReached": 76.44,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:15:38.311Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "311.7",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 326824240,
    "versionId": "darknet53"
},{
    "accuracyReached": 77.52,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-08-18T20:16:17.364Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "152.8",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 160242408,
    "versionId": "darknet19"
},{
    "accuracyReached": 77.1,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-11-23T07:40:58.799Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "28.6",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 29955696,
    "versionId": "cspdarknet_tiny"
},{
    "accuracyReached": 76.44,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-09-10T00:55:18.313Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "103.0",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 107993624,
    "versionId": "cspdarknet53"
},{
    "accuracyReached": 77.52,
    "batchSize": 1,
    "createdByUser": "n90fe0en2gvll5957fel7u75sg",
    "createdDate": "2021-09-10T00:55:09.000Z",
    "description": "",
    "gpuModel": "V100",
    "memoryFootprint": "62.9",
    "numberOfEpochs": 80,
    "status": "UPLOAD_COMPLETE",
    "totalFileCount": 1,
    "totalSizeInBytes": 65913984,
    "versionId": "cspdarknet19"
}]

which includes the efficientnet_b1_relu which did perform well when I used it … so am wondering if the table in the RetinaNet guide would be updated or perhaps I’ve just been lucky to succeed using that b1_relu model in a non-supported role?

14

also, interestingly have just come across this, which seems to explain why the efficeintnet_b1_relu would be expected to work –
Overview — TAO Toolkit 3.21.11 documentation (nvidia.com)

image
image4153×1766 117 KB

16

Sorry, after checking, B1~B5 are not supported for Retinanet. I can follow up with internal team for your request in future release.

17

Ok, thank you for the information, will not use B1-B5 any further for now, although the result of the B1 that was done seems to be valid so will leave that one experiment in our report for now. But won’t rely on it.

Thanks again for your help double checking all of those.

18

Do you mean you can run retinanet training successfully with B1 backbone?

19

correct

20

it was the only efficient net listed under the folder that is given by NVIDIA in the retinanet jupyter notebook

21

Could you share the training spec when you run with B1 backbone ?

22 
random_seed: 42
retinanet_config {
  aspect_ratios_global: "[1.0, 2.0, 0.5]"
  scales: "[0.045, 0.09, 0.2, 0.4, 0.55, 0.7]"
  two_boxes_for_ar1: false
  clip_boxes: false
  loss_loc_weight: 0.8
  focal_loss_alpha: 0.25
  focal_loss_gamma: 2.0
  variances: "[0.1, 0.1, 0.2, 0.2]"
  arch: "efficientnet_b0"
  n_kernels: 1
  n_anchor_levels: 1
  feature_size: 256
  freeze_bn: False
  freeze_blocks: 0
}
training_config {
  enable_qat: False
  pretrain_model_path: "/workspace/tao-experiments/retinanet10/pretrained_efficientnet_b1_relu/pretrained_object_detection_vefficientnet_b1_relu/efficientnet_b1_relu.hdf5"
  batch_size_per_gpu: 8
  num_epochs: 180
  n_workers: 2
  checkpoint_interval: 10
  learning_rate {
    soft_start_annealing_schedule {
      min_learning_rate: 4e-5
      max_learning_rate: 1.5e-2
      soft_start: 0.1
      annealing: 0.3
    }
  }
  regularizer {
    type: L1
    weight: 2e-5
  }
  optimizer {
    sgd {
      momentum: 0.9
      nesterov: True
    }
  }