Modular shared library for a custom classification model

So I looked at the nvdsinfer_customclassifierparser.cpp file that is located in the nvdsinfer_customparter folder and adjusted is as seen in the file below:
adjusted_custom_file.txt (4.2 KB)

Please note line 38 and 39, there you can see some labels-so I assume that these this shared library will only work for a model that with corresponding labels? I want to make a gender detection, so I’ve trained a gender detection model with two labels (Female and Male) and adjusted line 39 so that the labels rather reads (Female, Male). When I ran this, I received this error:

Failed to load config file: Key file contains line “Femail” which is not a key-value pair, group, or comment
** ERROR: <gst_nvinfer_parse_config_file:1319>: failed
0:00:00.592264657 4933 0x6117cf0 WARN nvinfer gstnvinfer.cpp:844:gst_nvinfer_start: error: Configuration file parsing failed
0:00:00.592284074 4933 0x6117cf0 WARN nvinfer gstnvinfer.cpp:844:gst_nvinfer_start: error: Config file path: /home/aizatron/repos/RnD-video-tracking/models/Onnx_gender/labels_GENDER_CLASSIFICATION.txt

Here is my labels file:
labels_GENDER_CLASSIFICATION.txt (11 Bytes)
And here is my config file for the SGIE:
sgie_config_YOLO_GENDER_CLASSIFICATION.txt (785 Bytes)

as the log shown, Configuration file parsing failed, gst_nvinfer_parse_config_file is opensource. can you add log to debug? seems it is related to Femail parsing.

How can I add logs for that?

And when I started to work on this again after the weekend, I could not replicate this error:
Failed to load config file: Key file contains line “Femail” which is not a key-value pair, group, or comment
ERROR: <gst_nvinfer_parse_config_file:1319>: failed
0:00:00.592264657 4933 0x6117cf0 WARN nvinfer gstnvinfer.cpp:844:gst_nvinfer_start: error: Configuration file parsing failed
0:00:00.592284074 4933 0x6117cf0 WARN nvinfer gstnvinfer.cpp:844:gst_nvinfer_start: error: Config file path: /home/aizatron/repos/RnD-video-tracking/models/Onnx_gender/labels_GENDER_CLASSIFICATION.txt

When I then evaluated the custom classifier parser and saw that the “NvDsInferParseCustomTfSSD” function is not present in the “nvdsinfer_customclassifierparser.cpp”. So if I set “parse-classifier-func-name=NvDsInferParseCustomTfSSD” then I get this error: “Failed to init classify-postprocessor because dlsym failed to get func NvDsInferParseCustomTfSSD pointer”. (And I chose “parse-classifier-func-name” because this is a classifier model).

Now config file looks like this:
sgie_config_YOLO_GENDER_CLASSIFICATION.txt (837 Bytes)

When I use this config file, the program is run, but it does again not seem as if it is interacting with the streams…

you can add log in gst_nvinfer_parse_config_file.
what do you mean about “if it is interacting with the streams…”? can the detector model work? can you see the bounding boxes? you can add log in NvDsInferClassiferParseCustomSoftmax to check if there is classification model output.

So it turns out that yolo is generally only made for detection models, and that is why it did not work for classification (I should of researched this a bit more before starting), but I then used the yolov8n-cls.pt model and I can see that it is addling labels to the streams.

1 Like

I can at least see that the classification model is having an influence, and adding labels to the streams, but I got this warning while the engine was built:
WARNING: …/nvdsinfer/nvdsinfer_model_builder.cpp:1208 INT8 calibration file not specified. Trying FP16 mode.
WARNING: [TRT]: Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
WARNING: [TRT]: Using PreviewFeature::kFASTER_DYNAMIC_SHAPES_0805 can help improve performance and resolve potential functional issues.
WARNING: [TRT]: TensorRT encountered issues when converting weights between types and that could affect accuracy.
WARNING: [TRT]: If this is not the desired behavior, please modify the weights or retrain with regularization to adjust the magnitude of the weights.
WARNING: [TRT]: Check verbose logs for the list of affected weights.
WARNING: [TRT]: - 23 weights are affected by this issue: Detected subnormal FP16 values.
WARNING: [TRT]: - 1 weights are affected by this issue: Detected values less than smallest positive FP16 subnormal value and converted them to the FP16 minimum subnormalized value.

Should I retrain the initial onnx file with certain configuration changes?

please ignore these TensorRT warning logs.

Really? Because the gender classification model labels every person as “Male”, I assumed this happened because the weights has been altered…

how did you get the gender model? can the model give two labels by test tools?

I trained the model myself with this script:

from ultralytics import YOLO
model = YOLO('yolov8n-cls.pt')
model.train(task=classify, data = "path/to/folder/", epochs=1) #The training dataset consists of 40 000 images 
model = model.export(format=onnx)

When I test the model with two images then I get successful classifications. The labels of the classification model is [‘female’, ‘male’]. When I run the model in deepstream then I get those warnings, and all of the classifications that the model makes is male. When I added more logs to the shared library, I see that it always predicts a 0 confidence for female, and always a 1 confidence for male…

please check if it is preprocess issue. please make sure the DeepStream 's parameters are the same with the test tool’s.

How can I check if it is a preprocessing issue? And by deepstream parameters, are you refering to the config file that is inserted as a property in the inference plugin? (This file : sgie_config_YOLO_GENDER_CLASSIFICATION.txt (703 Bytes))

  1. please understand which parameters are needed for preprocess in the test tool. then set corresponding parameters in DeepStream’s configuration. for example, maintain-aspect-ratio, symmetric-padding, etc. please refer to the nvinfer’s parameters explanation.
  2. if still have problem, please use this method to dump the inference inputs, then compare with test tool’s.

Sorry but what is the test tool?

sorry, I mean the tool or script that can verify the model can work. you need to know how it do preprocess and what parameters are set.

Okay cool, I initially trained the model on my windows computer, and the deepstream software is running on my ubuntu computer. I then trained the model on my ubuntu and it worked then-not sure why the model would not work if it was on my windows…

Sorry for the late reply, Is this still an DeepStream issue to support? Thanks!

I trained the model on my ubuntu computer and exported it to onnx with Ultralytics, but then the model was inaccurate. The model is classifying images and it is adding meta data to the buffer though.

I am not sure why the model becomes inaccurate when Deepstream converts the Onnx model to a tensorRT model… I think it might either be because the pipeline is using RGB color format, while the model might have been trained to detect BGR color format…The other thing I think might work is if I use the torch package to export the Pytorch model to onnx, instead of Ultralytics…

Any recommendations?

There is no update from you for a period, assuming this is not an issue any more. Hence we are closing this topic. If need further support, please open a new one. Thanks.

  1. noticing the test code works fine, you need to understand how the code do preprocessing. then check if if deepstream supports all the preprocessing parameters.
  2. if still have problem, please use this method to dump the inference inputs, then compare with test code’ s.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.