How to deploy custom audio model classificator on deepstream?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Hardware Platform (Jetson / GPU) Orin NX
• DeepStream Version 7.1
• JetPack Version (valid for Jetson only) 36.4
• TensorRT Version 10.3.0.30
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hello,

I am trying to integrate a custom audio classification model into DeepStream, but I am experiencing discrepancies in inference results compared to those obtained in the Colab environment.

Model details:
Model: ResNet50 trained on log-Mel spectrograms (128 mel, n_fft=2048, hop=512, top_db=80)

Format: ONNX (FP32), input shape [1,1,635,128]. (same as SONYC in the samples)

Preprocessing in Colab: normalization with mean and std calculated on the dataset

DeepStream configuration:

Plugin: nvinferaudio

audio-framesize=220050 (5s)

audio-hopsize=44100 (1s)

audio-transform=melsdb

net-scale-factor=1.0

offsets=0.0

Despite aligning the preprocessing parameters between Colab and DeepStream, the predictions obtained in DeepStream are inconsistent with those in Colab.

How can I correctly export my audio classification model for use in DeepStream, ensuring that preprocessing and normalization are handled consistently between the training and inference environments?

Thank you in advance for your assistance.

@fanzh @dusty_nv

3

Do you mean you want to convert these information into nvinferaudio preprocessing parameters? If so, please refer to How audio-transform works?

4

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks