Controlling by offline voice command a bluethoot device and servo-moteur using rasbbery pico connected to jetson nano

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

hellooo i’m trying to control a servo moteur connected to a pico and the pico will be connected to jetson usiing thonny. I succefully connected a pico to my jetson nano and blink a led using thonny. The problem now is that I need to use thonny from code oss or lauch thonny using code oss cause the offline voice command will use jetson nano.
so for example from the offline voice command code on jetson code oss i will say open(servo) and the code oss will launch thonny , open and run the servo code

How to install thonny

5

Hi,

Could you share more details about the use case?

Do you want a speech recognition feature on Nano?
If yes, you can check the following tutorial:

Thanks.

6

hello AastaLLL thank you for responding so i tried to use jetson voice but i cannot run the docker/run.sh it said no directory and
dustynv/jetson-voice:r32.7.1 don’t work cause i have jetpack 4.6.3 when i tried to change it with my r32.7.3 it did not work i tried picovoice but i’m new at programation so I don’t know how to train cause it really have to be offline and to make the whole code i’m even more lost . I know i have to use rhino for voice command and that i can train it online but i don’t know how to make a code with that and a code that will run thonny cause the servo will be connected to pico gpio

7

Hi,

Suppose you can use r32.7.1 with JetPack 4.6.3.
Do you meet any errors? Could you share the output log with us?

Thanks.

8

yes i i’m new at linux and programation so i don’t know if i aplly those command in the right files

image_2023-04-12_014358385
image_2023-04-12_014358385698×458 12.2 KB

9

Hi,

Could you try the command:

./docker/run.sh --container dustynv/jetson-voice:r32.7.1

Thanks.

10

hello sorry for the late answer for the other topic too i tried to download python 3.7 into a environement for tensor flow and my jetson crash i cannot even open it now i will try to reflash it i will come soon to try thank you

12

hello AastaLLL my linux pc had problems so i had to the use sd version the code work thank you verryy much but now when i tri to run asr.py i receive :raise ValueError(‘either wav or mic argument must be specified’)
ValueError: either wav or mic argument must be specified i tried to look in google but nothing on it

13

i tried my mic using arecord -D plughw:2,0 -f S16_LE -r 48000 -c 2 test.wav and my mic work so i don’t know what must be the problem

14

root@jarvis-desktop:/jetson-voice/examples# python3 asr.py
Namespace(debug=False, default_backend=‘tensorrt’, global_config=None, list_devices=False, list_models=False, log_level=‘info’, mic=None, model=‘quartznet’, model_dir=‘/jetson-voice/data/networks’, model_manifest=‘/jetson-voice/data/networks/manifest.json’, profile=False, verbose=False, wav=None)
[NeMo W 2023-04-15 02:52:30 nemo_logging:349] /usr/local/lib/python3.6/dist-packages/pydub/utils.py:170: RuntimeWarning: Couldn’t find ffmpeg or avconv - defaulting to ffmpeg, but may not work
warn(“Couldn’t find ffmpeg or avconv - defaulting to ffmpeg, but may not work”, RuntimeWarning)

################################################################################

WARNING, path does not exist: KALDI_ROOT=/mnt/matylda5/iveselyk/Tools/kaldi-trunk

(please add ‘export KALDI_ROOT=<your_path>’ in your $HOME/.profile)

(or run as: KALDI_ROOT=<your_path> python <your_script>.py)

################################################################################

[NeMo I 2023-04-15 02:52:30 features:264] PADDING: 0
[NeMo I 2023-04-15 02:52:30 features:281] STFT using torch
[NeMo W 2023-04-15 02:52:30 nemo_logging:349] /usr/local/lib/python3.6/dist-packages/nemo_toolkit-1.6.2-py3.6.egg/nemo/collections/asr/parts/preprocessing/features.py:314: FutureWarning: Pass sr=16000, n_fft=512 as keyword args. From version 0.10 passing these as positional arguments will result in an error
librosa.filters.mel(sample_rate, self.n_fft, n_mels=nfilt, fmin=lowfreq, fmax=highfreq), dtype=torch.float

[2023-04-15 02:52:31] resource.py:114 - loading model ‘/jetson-voice/data/networks/asr/quartznet-15x5_en/quartznet.onnx’ with jetson_voice.backends.tensorrt.TRTModel
[2023-04-15 02:52:31] trt_model.py:41 - loading cached TensorRT engine from /jetson-voice/data/networks/asr/quartznet-15x5_en/quartznet.engine
[04/15/2023-02:52:35] [TRT] [I] [MemUsageChange] Init CUDA: CPU +225, GPU +0, now: CPU 333, GPU 3573 (MiB)
[04/15/2023-02:52:35] [TRT] [I] Loaded engine size: 42 MiB
[04/15/2023-02:52:37] [TRT] [V] Using cublas as a tactic source
[04/15/2023-02:52:37] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +158, GPU +207, now: CPU 535, GPU 3869 (MiB)
[04/15/2023-02:52:37] [TRT] [V] Using cuDNN as a tactic source
[04/15/2023-02:52:41] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +240, GPU +20, now: CPU 775, GPU 3889 (MiB)
[04/15/2023-02:52:41] [TRT] [V] Deserialization required 5362705 microseconds.
[04/15/2023-02:52:41] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +41, now: CPU 0, GPU 41 (MiB)
[04/15/2023-02:52:41] [TRT] [V] Using cublas as a tactic source
[04/15/2023-02:52:41] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +1, GPU +0, now: CPU 733, GPU 3847 (MiB)
[04/15/2023-02:52:41] [TRT] [V] Using cuDNN as a tactic source
[04/15/2023-02:52:41] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +0, GPU +0, now: CPU 733, GPU 3847 (MiB)
[04/15/2023-02:52:41] [TRT] [V] Total per-runner device persistent memory is 36948992
[04/15/2023-02:52:41] [TRT] [V] Total per-runner host persistent memory is 282384
[04/15/2023-02:52:41] [TRT] [V] Allocated activation device memory of size 1076224
[04/15/2023-02:52:41] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +36, now: CPU 0, GPU 77 (MiB)
[2023-04-15 02:52:41] trt_model.py:59 - loaded TensorRT engine from /jetson-voice/data/networks/asr/quartznet-15x5_en/quartznet.engine

binding 0 - ‘audio_signal’
input: True
shape: (1, 64, -1)
dtype: DataType.FLOAT
size: -256
dynamic: True
profiles: [{‘min’: (1, 64, 10), ‘opt’: (1, 64, 150), ‘max’: (1, 64, 300)}]

binding 1 - ‘logprobs’
input: False
shape: (1, -1, 29)
dtype: DataType.FLOAT
size: -116
dynamic: True
profiles:

[2023-04-15 02:52:42] ctc_beamsearch.py:51 - creating CTCBeamSearchDecoder
[2023-04-15 02:52:42] ctc_beamsearch.py:52 - {‘add_punctuation’: True,
‘alpha’: 0.7,
‘beam_width’: 32,
‘beta’: 0.0,
‘cutoff_prob’: 1.0,
‘cutoff_top_n’: 40,
‘language_model’: ‘/jetson-voice/data/networks/asr/quartznet-15x5_en/lm.bin’,
‘timestep_offset’: 5,
‘top_k’: 3,
‘type’: ‘beamsearch’,
‘vad_eos_duration’: 0.65,
‘word_threshold’: -1000.0}
[2023-04-15 02:52:49] asr_engine.py:128 - CTC decoder type: ‘beamsearch’
Traceback (most recent call last):
File “asr.py”, line 30, in
chunk_size=asr.chunk_size)
File “/jetson-voice/jetson_voice/utils/audio.py”, line 67, in AudioInput
raise ValueError(‘either wav or mic argument must be specified’)
ValueError: either wav or mic argument must be specified

15

its ok dusty_nv answered my question but could you explain me please how to retrain de model he send me a link and told me to follow but i’m really lost i’m really new at this and don"t know but what i have to begin

16

Hi,

Sorry for the late update.

Do you want to retrain the model in jetson-voice?
If yes, the model is trained with NeMo. You can find more information below:

Thanks.

17

hello sorry for the late reply and thank you i will study this closely

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.