• Hardware Platform Ubuntu 20.04 with Nvidia GeForce GTX 1080 Ti • DeepStream 6.3-triton-multiarch Docker image • TensorRT 8.5.3 • NVIDIA GPU Driver version 555.42.02
Problem Description: I am experiencing an issue with the NVIDIA DeepStream framework concerning the dimensionality of my custom segmentation model. After converting my det_model.onnx file to det_model.engine, DeepStream parses the engine file and infers the dimensions as (360, 640). However, this format does not include the channel depth, which is critical for DeepStream to function correctly.
For proper operation, DeepStream requires the input dimensions to be in the format (batch_dims,channel_depth, 360, 640). My engine file currently lacks this channel depth information.
I understand that this issue may not be a fault of DeepStream itself but rather relates to how my model was constructed and trained by someone else. Unfortunately, I only have access to the ONNX and Torch saved model files and do not have the flexibility to retrain the model to adjust the input dimensions. The original input shape of my model is (batch_size, 360, 640), which also needs to include the channel depth.
Could someone please advise me on how to ensure that the channel depth is included in the engine file during the conversion process? Are there specific settings or configurations I need to adjust in either the ONNX model or the DeepStream configuration files to resolve this issue?
Previously, I tried to include the channel depth in my ONNX model, but there are resize nodes immediately following the input layer. Unfortunately, these resize nodes are not scaling correctly due to the specified coefficient values, such as cubic_coeff_a = -0.75. As a result, the engine file generation fails from the updated ONNX file when I attempt to include the channel depth information.
Although I can generate a new ONNX file with the modification of adding the channel depth, performing inference on this new file results in garbage values.
If your model have no channel, the nvinfer cannot support that well. Maybe you can try to use nvdspreprocess to customize your own tensort data for nvinfer. You can refer to our demo sources\apps\sample_apps\deepstream-preprocess-test to learn how to use that.
Okay, I will look into this, but the question is the same, even if I preprocess the input stream, is it possible for me to modify input dims of the stream e.g 3,1280x720 to 360x640 which will be required by the model?
Yes. You can use nvdspreprocess to customize the tensor data from any dims of your model. Like our demo deepstream-pose-classification, the input layer is (N, C, T, V, M). You need to implement that yourself.
There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks