I am using slowfast pretrained model from mmaction to get features then pass it through a classifier,
so my first question is that the model needs a specific preprocessing on the input frames, how and where could i implement the preprocessing code on those frames before it pass through the engine,
second question is how could I pass the features i got by this model to another model (classifier model)?
The inference plugin nvinfer supports the pre-processing of scaling, color format transforming, normalization, … What kind of pre-processing do you need?
@Fiona.Chen thanks
So how could I apply those preprocesses on the frame before make it pass through the model?
And the model is channel first, how could I make the frames channel first also?
All the configuration is done in nvinfer configure file. Take deepstream-test1 for example, the file “dstest1_pgie_config.txt” is the nvinfer configure file. Firstly, you need to set correct “network-type”. Then you need to set correct model path and name, for different models, you need to set different parameter. “model-file” is for caffemodel, “uff-file” is for uff model, …
Only supports UFF model channel order setting now. Please refer to “uff-input-order”
The following parameters are related to pre-processing:
“net-scale-factor” is for normalization
“uff-input-dims” is for scaling, nvinfer will read onnx model dimension automatically, so no parameter is needed for onnx model.
“model-color-format” is for color format transform.
All the above parameters can be found in Gst-nvinfer — DeepStream 6.1.1 Release documentation
1 Like