Deepstream 7.1 NvDsPreprocess incompatible with FP16 Grayscale model

[Kaladin]

Hi,

After updating our Orin board to deepstream 7.1 we noticed our pipeline was generating a segmentation fault when running same configuration as deepstream 6.3.

After diving into the deepstream header files we found the issue:

Function NvDsPreProcessStatus NvDsPreProcessTensorImpl::prepare_tensor now generates two convert function calls, convertFcn and convertFcnHalf depending if the model is fp16 or fp32, however, for grayscale models only convertFcn is assiged to NvDsPreProcessConvert_C1ToP1Float, there’s no NvDsPreProcessConvert_C1ToP1Half function so when later on the function tries to call convertFcnHalf, segfault happens because it is null.

After going back to 6.3 we noticed NvDsPreProcessConvert_C1ToP1Float was called for both f16 and 32, we tried doing that but the model isnt processed properly.

Was support for fp16 grayscale models deprecated? is there a way to emulate how this worked in 6.3?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Nvidia Orin
• DeepStream Version 7.1
• JetPack Version (valid for Jetson only) 6.2
• TensorRT Version 10.5
• NVIDIA GPU Driver Version (valid for GPU only) 540

[fanzh]

@Kaladin

here’s no NvDsPreProcessConvert_C1ToP1Half function so when later on the function tries to call convertFcnHalf, segfault happens because it is null.

Thanks for the sharing! AKY, It is because NvDsPreProcessConvert_C1ToP1Half is not implemented. Here is a fix for DS7.1. nvdspreprocess_lib.zip (7.4 KB). Since gst-nvdspreprocess is opensource, you can rebuild and replace the so.
could you share the model and cfgs to verify the fix? Thanks! You can use forum private email. Please click forum avatar → "personal messages’ → “view all personal messages” → “new message”.

[fanzh]

This issue has been fixed internally. Hence the latter versions will include the fix. Closing this topic.