Two channels of audio input with ML models

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hi,

I am new to Jetson Nano. I need to use two channels of audio input on the board for my project.

Based on the link here Models — NVIDIA NeMo, MarbleNet (VAD) can be used for VAD. I am interested in this and have the following questions （I hope to design a OWN VAD device）：

  1. Can jetson nano connect to a binaural analog mic?
    2）Is there any existing examples of how to run two ML models (one for each audio channel) simultaneously on Jetson Nano?
  2. Is there any existing models for speaker identification recognition (to realize the OWN part)? I think it should be similar to MarbleNet since they all use MFCC features.

Thank you in advance for any help.

Best,

Larry

3

Hi,

I just realized that the MarbleNet can be modified so that it can be used for OWN VAD. Maybe a Gaussian Mixture Model can be added later on for speaker recognition. Is it correct?

In this one, only one channel of audio input is needed. I only need to figure out how to modify the MarbleNet to complete the speaker recognition part. Any similar project before?

Thank you in advance for any help.

Best,

Larry

4

Hi,

Please check if the below repo can meet your requirement or not:

Thanks.

5

Hi,

Thank you for your kind reply. I am new to this area. Maybe my questions are stupid.

  1. Can I simulate the performance of the above inference library first offline (on a PC)? Because I hope to modify the MarbleNet later on in my project.
  2. For the hardware connection part, since Nvidia Jetson Nano has 4 USB ports. For implementing audio input, the only thing I need to purchase is a USB mic, is it correct? If I need two channels mic input, I just need two USB mics and insert them into USB port directly? If this is correct, does it mean Jetson Nano can support up to 4 audio input with the usage of usb mics?

Thanks in advance.

Larry

6

Sorry, I have another question apart from the previously mentioned two questions. Do you know what is the estimated on device performance of the above inference? Can the inference time be limited to 10ms? I am wondering whether jetson nano is fast enough to run these models. Thanks.

7

Hi,

Sorry for the late update.

1. Since TensorRT is hardware dependent, you will need to test it directly on the device.
2. Please check the below topic for the info:

3. Sorry that we don’t have a perf number for Nano.
But since Nano is quite limited in resources, you will need a lightweight model to reach the 10ms performance.

Thanks.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.