Trainning a custom resnet18 classificator and deploy it in deepstream as a secondary model

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) AGX ORIN
• DeepStream Version 6.1
• JetPack Version (valid for Jetson only) 5.0

Which framework should I use to train a custom classifier and export it to deploy it with deepstream as a sgie?

I am looking for a light model such as resnet18 for example. I guess that it would be optimal if i managed to calibrate it to use the int8 precision so it can run “fast” event with a batch size of, say, 16

Thanks!

4

@aurelm95
Not sure if you are using TAO. Can you share the info?

5

@Morganh
No, I am not using TAO.

I want to do the same as test2 from deepstream_python_apps does, that is, deploying a custom resnet18 classifier as a sgie.

So I would like to know how I should train it and how I should export accordingly so deepstream can parse its output.

If the easiest way to do so is using TAO, then I will use it.

9

@Morganh Any idea about how to proceed?

10

@aurelm95 DS team will help you check further. I am focusing on TAO forum to address tao related topics. Sorry for inconvenient.
And actually you can use TAO to train a classification resnet18 model. You can get started with running some notebooks. TAO Toolkit Quick Start Guide — TAO Toolkit 4.0 documentation

11

DeepStream is only a deploy SDK. You may need to choose the training tool by yourself. For model deployment, you’ve already read the python test2 sample, what you have to do is just to choose the PGIE and SGIE models and fill the nvinfer configuration file according to your models.

12

Okay so now I have my reset18 classification model. I have not been able to properly modify the config to see the output of the model.

Here you can see a screenshot of its netron visualization:

netron
netron1228×875 32.4 KB

I used this command to convert it to engine:

/usr/src/tensorrt/bin/trtexec --onnx=resnet18.onnx --saveEngine=resnet18.trt --workspace=8192 --minShapes=actual_input:1x3x244x244 --optShapes=actual_input:1x3x244x244 --maxShapes=actual_input:16x3x244x244

This is my current config file:

# Mandatory properties for classifiers:
#   classifier-threshold, is-classifier
#
# Optional properties for classifiers:
#   classifier-async-mode(Secondary mode only, Default=false)
#
# Optional properties in secondary mode:
#   operate-on-gie-id(Default=0), operate-on-class-ids(Defaults to all classes),
#   input-object-min-width, input-object-min-height, input-object-max-width,
#   input-object-max-height
#
# Following properties are always recommended:
#   batch-size(Default=1)
#
# Other optional properties:
#   net-scale-factor(Default=1), network-mode(Default=0 i.e FP32),
#   model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path,
#   mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary),
#   custom-lib-path, network-mode(Default=0 i.e FP32)
#
# The values in the config file are overridden by values set through GObject
# properties.

[property]
gpu-id=0
net-scale-factor=1

model-engine-file=/home/jetson/Desktop/resnet/resnet18.trt
labelfile-path=/home/jetson/Desktop/resnet/imagenet1k_labels.txt

force-implicit-batch-dim=1
batch-size=16
# 0=FP32 and 1=INT8 mode
network-mode=0
input-object-min-width=64
input-object-min-height=64
process-mode=2
model-color-format=1
gpu-id=0
gie-unique-id=2
operate-on-gie-id=1
operate-on-class-ids=0
is-classifier=1
output-blob-names=output
classifier-async-mode=1
classifier-threshold=0.51
process-mode=2
#scaling-filter=0
#scaling-compute-hw=0

This is what i get in the terminal:

Creating Pipeline 
 
Creating Source 
 
Creating H264Parser 

Creating Decoder 

Creating EGLSink 

Playing file /opt/nvidia/deepstream/deepstream-6.1/samples/streams/sample_720p.h264 
Adding elements to Pipeline 

Linking elements in the Pipeline 

Starting pipeline 


Using winsys: x11 
Opening in BLOCKING MODE 
0:00:02.536453974 132508     0x2f78f8a0 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<resnet18> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1909> [UID = 2]: deserialized trt engine from :/home/jetson/Desktop/resnet/resnet18.trt
INFO: [FullDims Engine Info]: layers num: 2
0   INPUT  kFLOAT actual_input    3x244x244       min: 1x3x244x244     opt: 1x3x244x244     Max: 16x3x244x244    
1   OUTPUT kFLOAT output          1000            min: 0               opt: 0               Max: 0               

0:00:02.683842186 132508     0x2f78f8a0 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<resnet18> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2012> [UID = 2]: Use deserialized engine model: /home/jetson/Desktop/resnet/resnet18.trt
0:00:02.721958460 132508     0x2f78f8a0 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<resnet18> [UID 2]: Load new model:resnet_config.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
[NvMultiObjectTracker] Initialized
0:00:02.768080515 132508     0x2f78f8a0 WARN                 nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1170> [UID = 1]: Warning, OpenCV has been deprecated. Using NMS for clustering instead of cv::groupRectangles with topK = 20 and NMS Threshold = 0.5
WARNING: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.1/sources/deepstream_python_apps/apps/deepstream-test2/../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine open error
0:00:04.404679868 132508     0x2f78f8a0 WARN                 nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1897> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-6.1/sources/deepstream_python_apps/apps/deepstream-test2/../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine failed
0:00:04.553388733 132508     0x2f78f8a0 WARN                 nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2002> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-6.1/sources/deepstream_python_apps/apps/deepstream-test2/../../../../samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine failed, try rebuild
0:00:04.553445214 132508     0x2f78f8a0 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1923> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: The implicit batch dimension mode has been deprecated. Please create the network with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag whenever possible.
WARNING: Serialize engine failed because of file path: /opt/nvidia/deepstream/deepstream-6.1/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine opened error
0:00:56.946004771 132508     0x2f78f8a0 WARN                 nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1950> [UID = 1]: failed to serialize cude engine to file: /opt/nvidia/deepstream/deepstream-6.1/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine
INFO: [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT input_1         3x368x640       
1   OUTPUT kFLOAT conv2d_bbox     16x23x40        
2   OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40         

0:00:57.108272558 132508     0x2f78f8a0 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:dstest2_pgie_config.txt sucessfully
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Frame Number=0 Number of Objects=0 Vehicle_count=0 Person_count=0
Frame Number=1 Number of Objects=0 Vehicle_count=0 Person_count=0
Frame Number=2 Number of Objects=0 Vehicle_count=0 Person_count=0
.
.
.

Is this

deserialized trt engine from :/home/jetson/Desktop/resnet/resnet18.trt
INFO: [FullDims Engine Info]: layers num: 2
0   INPUT  kFLOAT actual_input    3x244x244       min: 1x3x244x244     opt: 1x3x244x244     Max: 16x3x244x244    
1   OUTPUT kFLOAT output          1000            min: 0               opt: 0               Max: 0

correct??

Any hint on how to continue?

Thanks!

13

I see that sgie1 from the example has the following input/ouput shapes:

INFO: [Implicit Engine Info]: layers num: 2
0   INPUT  kFLOAT input_1         3x224x224       
1   OUTPUT kFLOAT predictions/Softmax 12x1x1

Should my model then have the following structure:

INFO: [FullDims Engine Info]: layers num: 2
0   INPUT  kFLOAT actual_input    3x244x244       min: 1x3x244x244     opt: 1x3x244x244     Max: 16x3x244x244    
1   OUTPUT kFLOAT output          1000x1x1            min: 0               opt: 0               Max: 0

If so, then how can I change the 1000 to 1000x1x1?

14

Okay I solved the problem. I had a bad formated label file and I did not notice it…

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.