Run any pretrained model : People net, Facenet etc. using webcam?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Nano
• DeepStream Version 5.1
• JetPack Version (valid for Jetson only) 4.5.1

Hi ,

The above is the hardware and versions i’m using in the Jetson Nano. I am trying to run the deepstream pretrained tlt-models using a webcam. I am able to run the FaceDetectIR model using the RTSP(ip cam) by editing the config. I also want to know the location of the source code of the FaceDetectIR so that I can make necessary configurations in the source code.

Im not able to run using webcam. Can anyone help me on this?

What do you mean? Do you want to modify the model?

You mentioned you can run the model. With which application?

Why do you say the following? What do you want to run?

Can you elaborate what are you talking about? The modle or the deepstream application?

Im trying to run the tlt model FaceDetectIR → https://ngc.nvidia.com/catalog/models/nvidia:tlt_facedetectir

I want to run this model in the deepstream application using webcam. Now i’m able to run this model over video files and RTSP cam using “deepstream-app -c deepstream_app_source1_faceirnet.txt” command, in which i changed the

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
num-sources=1
uri=rtsp://root:root321@10.20.1.3/axis-media/media.amp?streamprofile=H264

I want to know the steps to run the model in a deepstream app using webcam.

How does the webcam connected to the Nano board?
Most webcams are ip cameras with RTSP network connection. Webcam - Wikipedia
I’m confused with your description.

Hi,

It is a usb camera. I have connected it using USB

OK. Can you use “v4l2-ctl -d /dev/video0 --list-formats-ext” to query the camera information?

nano@nano-desktop-deepstream:~/Documents/FaceDetection$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘YUYV’
Name : YUYV 4:2:2
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 160x90
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 160x120
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 176x144
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 320x180
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 352x288
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 432x240
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 640x360
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 800x448
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 800x600
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 864x480
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.050s (20.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 960x720
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 1024x576
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 1600x896
Interval: Discrete 0.133s (7.500 fps)
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.200s (5.000 fps)
Size: Discrete 2304x1296
Interval: Discrete 0.500s (2.000 fps)
Size: Discrete 2304x1536
Interval: Discrete 0.500s (2.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'MJPG' (compressed)
Name        : Motion-JPEG
	Size: Discrete 640x480
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 160x90
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 160x120
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 176x144
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 320x180
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 320x240
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 352x288
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 432x240
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 800x448
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 800x600
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 864x480
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 960x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1024x576
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1600x896
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.042s (24.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)

So you can try V4L2 source. DeepStream Reference Application - deepstream-app — DeepStream 6.1.1 Release documentation

For example:
[source0]
enable=0
type=1
camera-width=1280
camera-height=720
camera-fps-n=10
camera-fps-d=1
camera-v4l2-dev-node=/dev/video0

Please read the document carefully. DeepStream Reference Application - deepstream-app — DeepStream 6.1.1 Release documentation

One more doubt is regarding the ID of the detection bounding box keeps on changing fast. Is there any way to stop the id change when the object is in the frame.?

For eg:
While detecting people or faces the id keeps on changing rapidly for a single person.

nano@nano-desktop-deepstream:/opt/nvidia/deepstream/deepstream-5.1/samples/configs/tlt_pretrained_models$ deepstream-app -c deepstream_app_source1_facedetectir.txt
** ERROR: <parse_source:602>: parse_source failed
** ERROR: <parse_config_file:532>: parse_config_file failed
** ERROR: main:627: Failed to parse config file ‘deepstream_app_source1_facedetectir.txt’
Quitting
App run failed

File runs into error after altering the config file as mentioned :

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=1
camera-width=1280
camera-height=720
camera-fps-n=10
camera-fps-d=1
camera-v4l2-dev-node=/dev/video0

For RTSP (Working Source)
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
num-sources=1
uri=rtsp://root:Glueck321@10.0.1.36/axis-media/media.amp?streamprofile=H264
gpu-id=0

Please help.

Please change “camera-v4l2-dev-node=/dev/video0” to “camera-v4l2-dev-node=0”.

Thanks it works.

We are using DeepStream’s python binding for out implementation and we are facing below stated issues.

  1. We want to modularize as model loading, inference, and post-processing. But these things are getting done in the backend C++ code which is a deprecated one. So We are unable to access the model variable and getting the frame-by-frame inference from the package.

  2. We need to build a single function-based inference from the deepStream model in python. As the current python pipeline is suitable for the video-wise inference demo.

  3. If somehow we can manage to get the model loading and inference part from the backend code, I hope we’ll be done with whatever we’re currently trying to get.

Hi shankarjadhav232,

Please help to open a new topic. Thanks

I have also asked to open a new topic and please see once