How to add Voice capability during inference in SSD Model

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello,

Hope you all are fine.

I have trained the SSD model using my own dataset for object detection.

I want to ask how to add voice capability like when it detects the object speaker should say the label of that object during inference.

Currently the command for inference in bash shell I am using for the Inference is their any argument to add also:

detectnet --model=models/dir/ssd-mobilenet.onnx --labels=models/dir/labels.txt --input-blob=input_0 --output-cvg=scores --output-bbox=boxes /dev/video0

3

Hi,

You can find several tutorial from our users.
For example: https://www.youtube.com/watch?v=AZ0QLYxAHlM

Thanks.

4

I am using docker and this tutorial is outdated.

Should I edit this file (https://github.com/dusty-nv/jetson-inference/blob/627b5890b49449573b7c1af8de22ce985fc395e4/c/detectNet.cpp)
in the docker to c++ text to speech library or can I use some kind of python library of Pyttsx3 add into the correct file of the detect net as I am unable to find the required file.

I have tried to edit the jetson-inference-python-examples detectnet.py by adding pysttx3, but the model is not found:

I have to submit my project till 15 June nay help would be appreciable as it’s my FYP

thank you

5

Hi @saifurrehman4114, if you are using pyttsx3, you would want to edit detectnet.py since pyttsx3 is also a Python library. If you are using the jetson-inference container, what you can do is make a copy of detectnet.py with your modifications, and then mount it into the container. This way you can more easily edit it:

# run this outside of container
$ mkdir ~/my_project
$ cp /path/to/jetson-inference/python/examples/detectnet.py ~/my_project/my_detectnet.py

# make edits to ~/my_project/my_detectnet.py

# run the container and mount in my_project
$ cd /path/to/jetson-inference
$ docker/run.sh -v ~/my_project:/my_project

Then from inside the container, you should be able to find your source file under /my_project

# run from inside the container
$ ls /my_project
$ python3 /my_project/my_detectnet.py

Did you install the pyttsx3 in the container first?

$ pip3 install pyttsx3
6

Hi @dusty_nv have installed pyttsx3 then in the my_project folder I made a python name in which I wrote a code related to that it works properly.

Code:

import pyttsx3
engine = pyttsx3.init()
engine.say(“I will speak this text”)
engine.runAndWait()

But this error show up.

Screenshot from 2021-06-08 07-54-19
Screenshot from 2021-06-08 07-54-191375Ă—876 166 KB

And also in the detect net file https://github.com/dusty-nv/jetson-inference/blob/627b5890b49449573b7c1af8de22ce985fc395e4/python/examples/detectnet.py#L63

I will return the objects detected as a list in the https://github.com/dusty-nv/jetson-inference/blob/627b5890b49449573b7c1af8de22ce985fc395e4/python/examples/detectnet.py#L68

After resolving pyttsx3

I can add in line 70 engine.say(detection) it will speak the name of the objects detected from the class or labels.txt names we have added.

Thanks alot

7

Sorry I interpreted your comment to mean that you had already fixed the problem - here is the solution:

apt-get update && apt-get install espeak
pip3 install pyttsx3
1 Like
8

Thanks, it’s running. My suggestion is to add this in jetson inference or in utils. Maybe use C++ lib for text to speech.

Thank you! for resolving the issue.