Optimizer for Image Classification

Hello
I am trying to train a model using TAO. In the documentation, I see that there are 3 optimizers that we can configure, but I do not see any info on which parameters to configure for the different optimizers.
The example document only has this.

# Only ['sgd', 'adam'] are supported for optimizer
  optimizer {
      sgd {
      lr: 0.01
      decay: 0.0
      momentum: 0.9
      nesterov: False
      }
  }

Can someone please tell which parameters need to be set to use ADAM?

NOTE: I am referring to the document here Image Classification — TAO Toolkit 3.21.11 documentation

Thanks.

You can refer to Multitask Image Classification — TAO Toolkit 3.21.11 documentation

optimizer {
adam {
epsilon: 1e-7
beta1: 0.9
beta2: 0.999
amsgrad: false
}
}

Ah! Thank you very much.

Sorry, when I try to run the config mentioned above, I am getting an error like:
google.protobuf.text_format.ParseError: 33:7 : Message type "AdamOptimizerConfig" has no field named "beta1"

After checking, the parameters for classification are:

message AdamOptimizerConfig {
float lr = 1;
float beta_1 = 2;
float beta_2 = 3;
float epsilon = 4;
float decay = 5;
}

So,

optimizer {
adam {
epsilon: 1e-7
beta_1: 0.9
beta_2: 0.999
lr: 0.01 (just an example)
decay: 0.0
}
}

Oh I see. Thanks.
One last question. Can you tell me where to check these specifications? (Like the protobuf message structure you mentioned above)

We will update the document. Current user guide is missing description of adam.

That would be great! Thanks alot.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.