Export Classifier to DeepStream

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

I am using TLT to train a classifier(resnet18) for OCR. I have 3 queries.

  1. How should the labels.txt in deepstream-test_app-2 look for a given classmap.json?
    Here is my classmap.json
{"1": 1, "0": 0, "3": 3, "2": 2, "5": 5, "4": 4, "7": 7, "6": 6, "9": 9, "8": 8, "A": 10, "C": 12, "B": 11, "E": 14, "D": 13, "G": 16, "F": 15, "H": 17, "K": 19, "J": 18, "M": 21, "L": 20, "O": 23, "N": 22, "Q": 25, "P": 24, "S": 27, "R": 26, "U": 29, "T": 28, "W": 31, "V": 30, "Y": 33, "X": 32, "Z": 34}

And this is my labels.txt

0;1;2;3;4;5;6;7;8;9;A;B;C;D;E;F;G;H;J;K;L;M;N;O;P;Q;R;S;T;U;V;W;X;Y;Z

.
But my results are quite random even though my validation accuracy was 99 during training. So the .engine file shouldnt be the problem. How should my labels.txt be? Can it have numbers? Is only lowercase allowed?

  1. Does the input size have to be a square? I get errors during evaluation if I try with different height and width i.e, has to be 224,224 and not 112,224.

  2. Is Data Augmentation possible for classification on TLT? If yes where to specify the parameters on the spec.cfg file?

2

Hi sharanssundar,

  1. According to tlt doc, there is no limitation for numbers or lowercase.
    The label file is a text file, containing the names of the classes that the TLT model is trained to classify against. The order in which the classes are listed must match the order in which the model predicts the output.
    Could you try deepstream-test_app-1?

  2. According to tlt doc, for Classification, the requirement is as below.
    Input size: 3 * H * W (W, H >= 16)
    Input format: JPG, JPEG, PNG

Can you paste your error log during evaluation?

  1. Yes. You can add more classes for your training. Just like the way reducing the classes, refer to https://devtalk.nvidia.com/default/topic/1065510/transfer-learning-toolkit/training-classification-model-from-scratch-/
3

Thank you Morganh. I will look up on the error logs. However,my 3rd question was about training with data augmentation like rotation,hue,saturation.etc and not just adding more classes. But I couldnt find any suitable terms in the spec file. So my question is, is data augmentation possible for training in TLT?

4

Hi sharanssundar,
Yes, TLT can train the augmentation dataset.
But you need to generate the augmentation dataset for each class by yourself.

NVIDIA Transfer Learning Toolkit is a Python package that enables NVIDIA customers to fine-tune pre-trained models with their own data.

5

Thanks Morgan for the clarification.

6

Hi sharanssundar,
For " Does the input size have to be a square? I get errors during evaluation if I try with different height and width i.e, has to be 224,224 and not 112,224." , there is an issue for non-square setting of input_image_size during “tlt-infer”. We will fix it. Thanks very much for the finding.

7

Hi Morganh. Glad to have contributed.