Deepstream 2.0 can not use USB camera

I make changes on configuration file as following

Copyright © 2018 NVIDIA Corporation. All rights reserved.

NVIDIA Corporation and its licensors retain all intellectual property

and proprietary rights in and to this software, related documentation

and any modifications thereto. Any use, reproduction, disclosure or

distribution of this software and related documentation without an express

license agreement from NVIDIA Corporation is strictly prohibited.

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
flow-original-resolution=1
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
width=960
height=540
gpu-id=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=1
camera-width=640
camera-height=480
camera-fps-n=30
camera-fps-d=1
camera-v4l2-dev-node=0

num-sources=1
gpu-id=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=1
source-id=0
gpu-id=0

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4

only SW mpeg4 is supported right now.

codec=3
sync=0
bitrate=2000000
output-file=out.mp4
source-id=0

[osd]
enable=1
gpu-id=0
osd-mode=1
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Arial
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0

[primary-gie]
enable=1
gpu-id=0
net-scale-factor=0.0039215697906911373
model-file=…/…/models/Primary_Detector/resnet10.caffemodel
proto-file=…/…/models/Primary_Detector/resnet10.prototxt
model-cache=…/…/models/Primary_Detector/resnet10.caffemodel_b16_fp32.cache
labelfile-path=…/…/models/Primary_Detector/labels.txt
net-stride=16
batch-size=4

0=FP32, 1=INT8, 2=FP16 mode

network-mode=0
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
num-classes=4
class-thresholds=0.2;0.2;0.1;0.2
class-eps=0.2;0.2;0.2;0.2
class-group-thresholds=1;1;1;1
roi-top-offset=0;0;0;0
roi-bottom-offset=0;0;0;0
detected-min-w=0;0;0;0
detected-min-h=0;0;0;0
detected-max-w=960;960;960;960
detected-max-h=540;540;540;540
interval=0
gie-unique-id=1
parse-func=4
output-bbox-name=conv2d_bbox
output-blob-names=conv2d_cov
parser-bbox-norm=35.0;35.0
#config-file=config_infer_resnet.txt

[tracker]
enable=1
tracker-width=640
tracker-height=368
gpu-id=0

[secondary-gie0]
enable=1
net-scale-factor=1
model-file=…/…/models/Secondary_VehicleTypes/resnet18.caffemodel
proto-file=…/…/models/Secondary_VehicleTypes/resnet18.prototxt
model-cache=…/…/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_fp32.cache
mean-file=…/…/models/Secondary_VehicleTypes/mean.ppm
labelfile-path=…/…/models/Secondary_VehicleTypes/labels.txt
gpu-id=0
batch-size=16
num-classes=6

0=FP32, 1=INT8, 2=FP16 mode

network-mode=0
detected-min-w=128
detected-min-h=128
detected-max-w=960
detected-max-h=540
model-color-format=1
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0;
is-classifier=1
output-blob-names=predictions/Softmax
classifier-async-mode=1
classifier-threshold=0.51

[secondary-gie1]
enable=1
net-scale-factor=1
model-file=…/…/models/Secondary_CarColor/resnet18.caffemodel
proto-file=…/…/models/Secondary_CarColor/resnet18.prototxt
model-cache=…/…/models/Secondary_CarColor/resnet18.caffemodel_b16_fp32.cache
mean-file=…/…/models/Secondary_CarColor/mean.ppm
labelfile-path=…/…/models/Secondary_CarColor/labels.txt
batch-size=16

0=FP32, 1=INT8, 2=FP16 mode

network-mode=0
detected-min-w=128
detected-min-h=128
detected-max-w=960
detected-max-h=540
model-color-format=1
num-classes=12
gpu-id=0
gie-unique-id=5
operate-on-gie-id=1
operate-on-class-ids=0;
is-classifier=1
output-blob-names=predictions/Softmax
classifier-async-mode=1
classifier-threshold=0.51

[secondary-gie2]
enable=1
net-scale-factor=1
model-file=…/…/models/Secondary_CarMake/resnet18.caffemodel
proto-file=…/…/models/Secondary_CarMake/resnet18.prototxt
model-cache=…/…/models/Secondary_CarMake/resnet18.caffemodel_b16_fp32.cache
mean-file=…/…/models/Secondary_CarMake/mean.ppm
labelfile-path=…/…/models/Secondary_CarMake/labels.txt
batch-size=16

0=FP32, 1=INT8, 2=FP16 mode

network-mode=0
num-classes=24
detected-min-w=128
detected-min-h=128
detected-max-w=960
detected-max-h=540
model-color-format=1
gpu-id=0
gie-unique-id=6
operate-on-gie-id=1
operate-on-class-ids=0;
is-classifier=1
output-blob-names=predictions/Softmax
classifier-async-mode=1
classifier-threshold=0.51

[tests]
file-loop=1

However, the console show the execution log, and I can’t see anything from camera side

$ deepstream-app -c configs/deepstream-app/camera_720p_dec_infer-resnet_tracker_sgie_tiled_display_fp32.txt
** WARN: <create_pipeline:888>: Num of Tiles less than number of sources, readjusting to 1 rows, 1 columns

Using Cached GIE model /home/michael/deepstream2.0/samples/configs/deepstream-app/…/…/models/Secondary_CarMake/resnet18.caffemodel_b16_fp32.cache crypto flags(0)
Using Cached GIE model /home/michael/deepstream2.0/samples/configs/deepstream-app/…/…/models/Secondary_CarColor/resnet18.caffemodel_b16_fp32.cache crypto flags(0)
Using Cached GIE model /home/michael/deepstream2.0/samples/configs/deepstream-app/…/…/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_fp32.cache crypto flags(0)
Using Cached GIE model /home/michael/deepstream2.0/samples/configs/deepstream-app/…/…/models/Primary_Detector/resnet10.caffemodel_b16_fp32.cache crypto flags(0)

Runtime commands:
h: Print this help
q: Quit

p: Pause
r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.

** INFO: <bus_callback:75>: Pipeline ready

** INFO: <bus_callback:60>: Pipeline running

nvvidconv: line=300 ---- video/x-raw(memory:NVMM), width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, framerate=(fraction)30/1, format=(string)NV12
nvstreamtiler: batchSize set as 4

**PERF: FPS 0 (Avg)
**PERF: -nan (-nan)
**PERF: -nan (-nan)
**PERF: -nan (-nan)

Any idea?

Hi Michael,
What is your camera output? raw YUV or encoded stream?
And how does your camera connect to PC?

Thanks
wayne zhu

YUV 4:2:2. The camera type is USB. Just connect camera to Desktop’s any USB 2.0/3.0 ports.

Hi Michael,

Can you check V4l2src plugin work well in your side?
If YUV422 input, you need to convert it to YUV420p before providing it to nvinfer/streammux.

Thanks
wayne zhu