Jetson nano rtsp stream failed with intel realsense 415 usb camera

Hi,

I am trying to run this on Jetson nano. but failed . could you please help?

Thanks,

deepstream-test5-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt

(deepstream-test5-app:23222): GStreamer-WARNING **: 01:09:46.139: Name ‘src_cap_filter’ is not unique in bin ‘src_sub_bin0’, not adding
Creating LL OSD context new
0:00:03.423538280 23222 0x298fc300 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:useEngineFile(): Failed to read from model engine file
0:00:03.423615209 23222 0x298fc300 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:03.423774431 23222 0x298fc300 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): INT8 not supported by platform. Trying FP16 mode.
0:02:12.964284941 23222 0x298fc300 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_fp16.engine

Runtime commands:
h: Print this help
q: Quit

p: Pause
r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.

**PERF: FPS 0 (Avg)
**PERF: 0.00 (0.00)
** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

ERROR from src_elem: Internal data stream error.
Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming stopped, reason not-negotiated (-4)
Quitting
App run failed

some kernel log hopefully helps

[ 8919.206660] ------------[ cut here ]------------
[ 8919.211277] WARNING: CPU: 3 PID: 24402 at /dvs/git/dirty/git-master_linux/kernel/kernel-4.9/drivers/media/v4l2-core/v4l2-ioctl.c:1305 v4l_enum_fmt+0x11dc/0x1548
[ 8919.225591] Modules linked in: hid_logitech_hidpp hid_logitech_dj fuse zram uvcvideo bnep overlay btusb btrtl btbcm btintel iwlmvm mac80211 iwlwifi cfg80211 nvgpu bluedroid_pm ip_tables x_tables

[ 8919.225634] CPU: 3 PID: 24402 Comm: cheese Tainted: G W 4.9.140-tegra #1
[ 8919.225636] Hardware name: NVIDIA Jetson Nano Developer Kit (DT)
[ 8919.225640] task: ffffffc0d002aa00 task.stack: ffffffc0d82ac000
[ 8919.225644] PC is at v4l_enum_fmt+0x11dc/0x1548
[ 8919.225647] LR is at v4l_enum_fmt+0x11dc/0x1548
[ 8919.225650] pc : [] lr : [] pstate: 40400145
[ 8919.225652] sp : ffffffc0d82afbc0
[ 8919.225654] x29: ffffffc0d82afbc0 x28: 0000000000000000
[ 8919.225660] x27: ffffffc0ae01e018 x26: 0000000000000000
[ 8919.225665] x25: ffffff8009faf480 x24: 0000000000000002
[ 8919.225670] x23: 00000055a2bc45b0 x22: ffffffc09636df00
[ 8919.225675] x21: ffffff80013fef28 x20: 0000000000000000
[ 8919.225680] x19: ffffffc0d82afd10 x18: 0000000000000030
[ 8919.225685] x17: 0000000000000000 x16: 0000000000000000
[ 8919.225690] x15: ffffffffffffffff x14: ffffff800a124260
[ 8919.225695] x13: 0000000000000000 x12: 0000000000000006
[ 8919.225700] x11: 0000000000000002 x10: 00000000000025b9
[ 8919.225704] x9 : 0000000000000001 x8 : ffffffc0fef94698
[ 8919.225709] x7 : 0000000000000000 x6 : ffffffc0ff002bf0
[ 8919.225714] x5 : ffffffc0ff002bf0 x4 : 0000000000000000
[ 8919.225719] x3 : ffffffc0ff0087f8 x2 : ffffffc0ff002bf0
[ 8919.225724] x1 : ffffffc0d002aa00 x0 : 000000000000001e

[ 8919.225730] —[ end trace 00da38cf4a50cf49 ]—
[ 8919.230337] Call trace:
[ 8919.230342] [] v4l_enum_fmt+0x11dc/0x1548
[ 8919.230346] [] __video_do_ioctl+0x204/0x2c8
[ 8919.230349] [] video_usercopy+0x2a0/0x6a0
[ 8919.230352] [] video_ioctl2+0x3c/0x50
[ 8919.230358] [] v4l2_ioctl+0xd0/0x118
[ 8919.230363] [] do_vfs_ioctl+0xb0/0x8d8
[ 8919.230366] [] SyS_ioctl+0x8c/0xa8
[ 8919.230371] [] el0_svc_naked+0x34/0x38
[ 8919.230513] Unknown pixelformat 0x49323159
[ 8919.230559] ------------[ cut here ]------------
[ 8919.235178] WARNING: CPU: 3 PID: 24402 at /dvs/git/dirty/git-master_linux/kernel/kernel-4.9/drivers/media/v4l2-core/v4l2-ioctl.c:1305 v4l_enum_fmt+0x11dc/0x1548
[ 8919.249491] Modules linked in: hid_logitech_hidpp hid_logitech_dj fuse zram uvcvideo bnep overlay btusb btrtl btbcm btintel iwlmvm mac80211 iwlwifi cfg80211 nvgpu bluedroid_pm ip_tables x_tables

[ 8919.249534] CPU: 3 PID: 24402 Comm: cheese Tainted: G W 4.9.140-tegra #1
[ 8919.249536] Hardware name: NVIDIA Jetson Nano Developer Kit (DT)
[ 8919.249539] task: ffffffc0d002aa00 task.stack: ffffffc0d82ac000
[ 8919.249543] PC is at v4l_enum_fmt+0x11dc/0x1548
[ 8919.249546] LR is at v4l_enum_fmt+0x11dc/0x1548
[ 8919.249549] pc : [] lr : [] pstate: 40400145
[ 8919.249552] sp : ffffffc0d82afbc0
[ 8919.249554] x29: ffffffc0d82afbc0 x28: 0000000000000000
[ 8919.249559] x27: ffffffc0ae01e018 x26: 0000000000000000
[ 8919.249564] x25: ffffff8009faf480 x24: 0000000000000002
[ 8919.249569] x23: 00000055a2d21f30 x22: ffffffc09636df00
[ 8919.249574] x21: ffffff80013fef28 x20: 0000000000000000
[ 8919.249579] x19: ffffffc0d82afd10 x18: 0000000000000030
[ 8919.249584] x17: 0000000000000000 x16: 0000000000000000
[ 8919.249589] x15: ffffffffffffffff x14: ffffff800a124260
[ 8919.249594] x13: 0000000000000000 x12: 0000000000000006
[ 8919.249599] x11: 0000000000000002 x10: 00000000000025df
[ 8919.249604] x9 : 0000000000000001 x8 : ffffffc0fef95010
[ 8919.249609] x7 : 0000000000000000 x6 : ffffffc0ff002bf0
[ 8919.249614] x5 : ffffffc0ff002bf0 x4 : 0000000000000000
[ 8919.249619] x3 : ffffffc0ff0087f8 x2 : ffffffc0ff002bf0
[ 8919.249623] x1 : ffffffc0d002aa00 x0 : 000000000000001e

[ 8919.249630] —[ end trace 00da38cf4a50cf4a ]—
[ 8919.254236] Call trace:
[ 8919.254241] [] v4l_enum_fmt+0x11dc/0x1548
[ 8919.254245] [] __video_do_ioctl+0x204/0x2c8
[ 8919.254249] [] video_usercopy+0x2a0/0x6a0
[ 8919.254252] [] video_ioctl2+0x3c/0x50
[ 8919.254257] [] v4l2_ioctl+0xd0/0x118
[ 8919.254263] [] do_vfs_ioctl+0xb0/0x8d8
[ 8919.254266] [] SyS_ioctl+0x8c/0xa8
[ 8919.254271] [] el0_svc_naked+0x34/0x38
[ 8919.254381] Unknown pixelformat 0x00000000
[ 8919.254435] ------------[ cut here ]------------
[ 8919.259057] WARNING: CPU: 3 PID: 24402 at /dvs/git/dirty/git-master_linux/kernel/kernel-4.9/drivers/media/v4l2-core/v4l2-ioctl.c:1305 v4l_enum_fmt+0x11dc/0x1548
[ 8919.273369] Modules linked in: hid_logitech_hidpp hid_logitech_dj fuse zram uvcvideo bnep overlay btusb btrtl btbcm btintel iwlmvm mac80211 iwlwifi cfg80211 nvgpu bluedroid_pm ip_tables x_tables

[ 8919.273412] CPU: 3 PID: 24402 Comm: cheese Tainted: G W 4.9.140-tegra #1
[ 8919.273414] Hardware name: NVIDIA Jetson Nano Developer Kit (DT)
[ 8919.273418] task: ffffffc0d002aa00 task.stack: ffffffc0d82ac000
[ 8919.273422] PC is at v4l_enum_fmt+0x11dc/0x1548
[ 8919.273425] LR is at v4l_enum_fmt+0x11dc/0x1548
[ 8919.273428] pc : [] lr : [] pstate: 40400145
[ 8919.273430] sp : ffffffc0d82afbc0
[ 8919.273433] x29: ffffffc0d82afbc0 x28: 0000000000000000
[ 8919.273440] x27: ffffffc0ae01e018 x26: 0000000000000000
[ 8919.273446] x25: ffffff8009faf480 x24: 0000000000000002
[ 8919.273452] x23: 00000055a2d8acc0 x22: ffffffc09636df00
[ 8919.273458] x21: 0000000000000000 x20: 0000000000000000
[ 8919.273464] x19: ffffffc0d82afd10 x18: 0000000000000030
[ 8919.273470] x17: 0000000000000000 x16: 0000000000000000
[ 8919.273476] x15: ffffffffffffffff x14: ffffff800a124260
[ 8919.273482] x13: 0000000000000000 x12: 0000000000000006
[ 8919.273487] x11: 0000000000000002 x10: 0000000000002605
[ 8919.273493] x9 : 0000000000000001 x8 : ffffffc0fef95988
[ 8919.273499] x7 : 0000000000000000 x6 : ffffffc0ff002bf0
[ 8919.273505] x5 : ffffffc0ff002bf0 x4 : 0000000000000000
[ 8919.273511] x3 : ffffffc0ff0087f8 x2 : ffffffc0ff002bf0
[ 8919.273517] x1 : ffffffc0d002aa00 x0 : 000000000000001e

[ 8919.273525] —[ end trace 00da38cf4a50cf4b ]—
[ 8919.278144] Call trace:
[ 8919.278149] [] v4l_enum_fmt+0x11dc/0x1548
[ 8919.278154] [] __video_do_ioctl+0x204/0x2c8
[ 8919.278157] [] video_usercopy+0x2a0/0x6a0
[ 8919.278161] [] video_ioctl2+0x3c/0x50
[ 8919.278167] [] v4l2_ioctl+0xd0/0x118
[ 8919.278172] [] do_vfs_ioctl+0xb0/0x8d8
[ 8919.278176] [] SyS_ioctl+0x8c/0xa8
[ 8919.278181] [] el0_svc_naked+0x34/0x38
[ 8932.328728] uvcvideo: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 8933.398322] uvcvideo: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
[ 8933.456871] uvcvideo: Failed to query (GET_CUR) UVC control 1 on unit 3: -32 (exp. 1024).
vaitl@tegra-ubuntu:/opt/nvidia/deepstream/deepstream-4.0/samples$

Hi,
Please share information about the camera so that we can suggest next.

$ v4l2-ctl -d /dev/video1 --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘’
Name : 00000032-0000-0010-8000-00aa003
Size: Discrete 256x144
Interval: Discrete 0.011s (90.000 fps)
Size: Discrete 424x240
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 480x270
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x360
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 848x100
Interval: Discrete 0.010s (100.000 fps)
Size: Discrete 848x480
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'UYVY'
Name        : UYVY 4:2:2
	Size: Discrete 256x144
		Interval: Discrete 0.011s (90.000 fps)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 848x100
		Interval: Discrete 0.010s (100.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)

Index       : 2
Type        : Video Capture
Pixel Format: 'GREY'
Name        : 8-bit Greyscale
	Size: Discrete 256x144
		Interval: Discrete 0.011s (90.000 fps)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 848x100
		Interval: Discrete 0.010s (100.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)

Index       : 3
Type        : Video Capture
Pixel Format: 'Y8I '
Name        : Greyscale 8 L/R (Y8I)
	Size: Discrete 256x144
		Interval: Discrete 0.011s (90.000 fps)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 848x100
		Interval: Discrete 0.010s (100.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.067s (15.000 fps)

Index       : 4
Type        : Video Capture
Pixel Format: 'Y12I'
Name        : Greyscale 12 L/R (Y12I)
	Size: Discrete 960x540
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
	Size: Discrete 1920x1080
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.067s (15.000 fps)

Index       : 5
Type        : Video Capture
Pixel Format: ''
Name        : 20303157-0000-0010-8000-00aa003
	Size: Discrete 256x144
		Interval: Discrete 0.011s (90.000 fps)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 848x100
		Interval: Discrete 0.010s (100.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)

Hi,
Configuring to ‘UYVY’ format should work, please check if you can run

$ gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,width=1280,height=720,framerate=30/1 ! videoconvert ! video/x-raw,format=I420 ! nvvideoncvert ! 'video/x-raw(memory:NVMM),format=I420' ! nvoverlaysink

I just tried it returns a err. btw how do i run with deepstream instead ?

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,width=1280,height=720,framerate=30/1 ! videoconvert ! video/x-raw,format=I420 ! nvvideoncvert ! ‘video/x-raw(memory:NVMM),format=I420’ ! nvoverlaysink
WARNING: erroneous pipeline: no element “nvvideoncvert”

i think you have a typo there … after i change to nvvideoconvert it runs. but how do i verify it ??
also how do i use deepstream app to do the same ??

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=UYVY,width=1280,height=720,framerate=30/1 ! videoconvert ! video/x-raw,format=I420 ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),format=I420’ ! nvoverlaysink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock

after running your command / i can see the camera on the screen.

now the question is that how to do i do the change with deepstream app to be able to send camear video to rstp remote .

thanks

Hi,
You have tried the sample config file in running deepstream-test5:

deepstream_sdk_v4.0.1_jetson\samples\configs\deepstream-app\source1_usb_dec_infer_resnet_int8.txt

Have you tried to run it in deepstream-app?

i tried . similar err . with this config file source1_usb_dec_infer_resnet_int8.txt it can not read the engine file defined model-engine-file=…/…/models/Primary_Detector/resnet10.caffemodel_b30_int8.engine

deepstream-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt

(deepstream-app:16087): GStreamer-WARNING **: 17:24:58.750: Name ‘src_cap_filter’ is not unique in bin ‘src_sub_bin0’, not adding

*** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***

Opening in BLOCKING MODE
Creating LL OSD context new
0:00:02.900897010 16087 0x10f08d90 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:useEngineFile(): Failed to read from model engine file
0:00:02.900994147 16087 0x10f08d90 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:02.901259726 16087 0x10f08d90 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): INT8 not supported by platform. Trying FP16 mode.
0:02:09.345437539 16087 0x10f08d90 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_fp16.engine

Runtime commands:
h: Print this help
q: Quit

p: Pause
r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.

**PERF: FPS 0 (Avg)
**PERF: 0.00 (0.00)
** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

ERROR from src_elem: Internal data stream error.
Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming stopped, reason not-negotiated (-4)
Quitting
App run failed

I got deepstream-test5-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt
working. but I have to do below two things

  1. Disable [primary-gie] becuase there is no resnet10 in part of deepstream sdk
    enable=0
    model-engine-file=…/…/models/Primary_Detector/resnet10.

2.I noticed that there are 3 dev node /dev/video0 /dev/video1/ /dev/video2
so in the config file source0 I tried camera-v4l2-dev-node=1 and camera-v4l2-dev-node=2 they both works . but only camera-v4l2-dev-node=0 is failing.

21 [source0]
22 enable=1
23 #Type - 1=CameraV4L2 2=URI 3=MultiURI
24 type=1
25 camera-width=1280
26 camera-height=720
27 camera-fps-n=30
28 camera-fps-d=1
29 camera-v4l2-dev-node=1 //Terry 2 also works

=================================================================================
deepstream-test5-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt

(deepstream-test5-app:25415): GStreamer-WARNING **: 23:05:22.440: Name ‘src_cap_filter’ is not unique in bin ‘src_sub_bin0’, not adding

*** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***

Opening in BLOCKING MODE
Creating LL OSD context new

Runtime commands:
h: Print this help
q: Quit

p: Pause
r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.

** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

ERROR from src_elem: Internal data stream error.
Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming stopped, reason not-negotiated (-4)
Quitting
App run failed

=================================================================================
here is v4l2-ctl dump on /dev/video0

v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'Z16 ’
Name : Depth data 16-bit (Z16)
Size: Discrete 256x144
Interval: Discrete 0.011s (90.000 fps)
Size: Discrete 424x240
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 480x270
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x360
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 848x100
Interval: Discrete 0.010s (100.000 fps)
Size: Discrete 848x480
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: ''
Name        : 00000050-0000-0010-8000-00aa003
	Size: Discrete 256x144
		Interval: Discrete 0.011s (90.000 fps)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 848x100
		Interval: Discrete 0.010s (100.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
  1. realsense camera provide depth image and rgb image i saw /dev/video2 is RGB image but i dont know how the depth image are create out of /dev/video0 and /dev/video1 . probably realsense sdk do some other processing to create the depth image. if that is the case. how do I plugin the realsense sdk driver into deepstream. should I write a application on top of realsense sdk . so instead deepstream read from /dev/video0 1 2 it read from my application as src into deepstream ?

Thanks,
Terry

Hi,
Please run ‘gst-inspect-1.0 v4l2src’ to get supported formats. It looks like the non-working formats are not supported by gstreamer frameworks. Since DS SDK is based on gstreamer, some device nodes on RealSense cameras may not be opened. A general usecase of USB cameras is to run in YUV422 such as UYVY, YUY2.

sure i will run it tomorrow . but could you tell me where to change the format in the configuration file ?

also if node is not opened as you suggested . Can I read the depth data from realsense application and send it to Deepstream as input ?

Hi,
By default we have the implementations in source group:
https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html#page/DeepStream_Development_Guide%2Fdeepstream_app_config.3.2.html%23wwpID0E0QB0HA

One possible solution is to add one type(ex: type=6) to support appsrc. In the implementation, you need to convert Z16 into NV12 and create source bin like

appsrc ! video/x-raw,format=NV12 ! nvvideoconvert ! 'video/x-raw(memory:NMM),format=NV12'

deepstream-app is open source and you can customize it into the usecase.
Other users may have other possible solutions to share here.

gst-inspect-1.0 v4l2src
Factory Details:
Rank primary (256)
Long-name Video (video4linux2) Source
Klass Source/Video
Description Reads frames from a Video4Linux2 device
Author Edgard Lima edgard.lima@gmail.com, Stefan Kost ensonic@users.sf.net

Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so
Version 1.14.5
License LGPL
Source module gst-plugins-good
Source release date 2019-05-29
Binary package GStreamer Good Plugins (Ubuntu)
Origin URL gst-plugins-good1.0 package : Ubuntu

GObject
±—GInitiallyUnowned
±—GstObject
±—GstElement
±—GstBaseSrc
±—GstPushSrc
±—GstV4l2Src

Implemented Interfaces:
GstURIHandler
GstTuner
GstColorBalance
GstVideoOrientation

Pad Templates:
SRC template: ‘src’
Availability: Always
Capabilities:
image/jpeg
video/mpeg
mpegversion: 4
systemstream: false
video/mpeg
mpegversion: 2
video/mpegts
systemstream: true
video/x-bayer
format: { (string)bggr, (string)gbrg, (string)grbg, (string)rggb }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-dv
systemstream: true
video/x-h263
variant: itu
video/x-h264
stream-format: { (string)byte-stream, (string)avc }
alignment: au
video/x-pwc1
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-pwc2
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { (string)RGB16, (string)BGR, (string)RGB, (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)YVU9, (string)YV12, (string)YUY2, (string)YVYU, (string)UYVY, (string)Y42B, (string)Y41B, (string)YUV9, (string)NV12_64Z32, (string)NV24, (string)NV61, (string)NV16, (string)NV21, (string)NV12, (string)I420, (string)BGRA, (string)BGRx, (string)ARGB, (string)xRGB, (string)BGR15, (string)RGB15 }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-sonix
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-vp8
video/x-vp9
video/x-wmv
wmvversion: 3
format: WVC1

Element has no clocking capabilities.

URI handling capabilities:
Element can act as source.
Supported URI protocols:
v4l2

Pads:
SRC: ‘src’
Pad Template: ‘src’

Element Properties:
name : The name of the object
flags: readable, writable
String. Default: “v4l2src0”
parent : The parent of the object
flags: readable, writable
Object of type “GstObject”
blocksize : Size in bytes to read per buffer (-1 = default)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 4096
num-buffers : Number of buffers to output before sending EOS (-1 = unlimited)
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
typefind : Run typefind before negotiating (deprecated, non-functional)
flags: readable, writable, deprecated
Boolean. Default: false
do-timestamp : Apply current stream time to buffers
flags: readable, writable
Boolean. Default: false
device : Device location
flags: readable, writable
String. Default: “/dev/video0”
device-name : Name of the device
flags: readable
String. Default: null
device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1
flags : Device type flags
flags: readable
Flags “GstV4l2DeviceTypeFlags” Default: 0x00000000, “(none)”
(0x00000001): capture - Device supports video capture
(0x00000002): output - Device supports video playback
(0x00000004): overlay - Device supports video overlay
(0x00000010): vbi-capture - Device supports the VBI capture
(0x00000020): vbi-output - Device supports the VBI output
(0x00010000): tuner - Device has a tuner or modulator
(0x00020000): audio - Device has audio inputs or outputs
brightness : Picture brightness, or more precisely, the black level
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
contrast : Picture contrast or luma gain
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
saturation : Picture color saturation or chroma gain
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
hue : Hue or color balance
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
norm : video standard
flags: readable, writable
Enum “V4L2_TV_norms” Default: 0, “none”
(0): none - none
(45056): NTSC - NTSC
(4096): NTSC-M - NTSC-M
(8192): NTSC-M-JP - NTSC-M-JP
(32768): NTSC-M-KR - NTSC-M-KR
(16384): NTSC-443 - NTSC-443
(255): PAL - PAL
(7): PAL-BG - PAL-BG
(1): PAL-B - PAL-B
(2): PAL-B1 - PAL-B1
(4): PAL-G - PAL-G
(8): PAL-H - PAL-H
(16): PAL-I - PAL-I
(224): PAL-DK - PAL-DK
(32): PAL-D - PAL-D
(64): PAL-D1 - PAL-D1
(128): PAL-K - PAL-K
(256): PAL-M - PAL-M
(512): PAL-N - PAL-N
(1024): PAL-Nc - PAL-Nc
(2048): PAL-60 - PAL-60
(16711680): SECAM - SECAM
(65536): SECAM-B - SECAM-B
(262144): SECAM-G - SECAM-G
(524288): SECAM-H - SECAM-H
(3276800): SECAM-DK - SECAM-DK
(131072): SECAM-D - SECAM-D
(1048576): SECAM-K - SECAM-K
(2097152): SECAM-K1 - SECAM-K1
(4194304): SECAM-L - SECAM-L
(8388608): SECAM-Lc - SECAM-Lc
io-mode : I/O mode
flags: readable, writable
Enum “GstV4l2IOMode” Default: 0, “auto”
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
extra-controls : Extra v4l2 controls (CIDs) for the device
flags: readable, writable
Boxed pointer of type “GstStructure”
pixel-aspect-ratio : Overwrite the pixel aspect ratio of the device
flags: readable, writable
String. Default: null
force-aspect-ratio : When enabled, the pixel aspect ratio will be enforced
flags: readable, writable
Boolean. Default: true

Element Signals:
“prepare-format” : void user_function (GstElement* object,
gint arg0,
GstCaps* arg1,
gpointer user_data);

Hi,
Don’t see Z16 is listed in

format: { (string)RGB16, (string)BGR, (string)RGB, (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)YVU9, (string)YV12, (string)YUY2, (string)YVYU, (string)UYVY, (string)Y42B, (string)Y41B, (string)YUV9, (string)NV12_64Z32, (string)NV24, (string)NV61, (string)NV16, (string)NV21, (string)NV12, (string)I420, (string)BGRA, (string)BGRx, (string)ARGB, (string)xRGB, (string)BGR15, (string)RGB15 }

Please refer to #15 on customizing deepstream-app.

Hi

Could you please explain a bit more.

Is it because realsense does not support z16 format or streamer does not support z16 format?

What it takes to add z16 format support?

Thanks

Hi,
I think gstreamer does not support z16 in v4l2src plugin. But maybe I am wrong since I don’t have experience to sun realsense camera in gstreamer pipeline.

Could you please go to
http://gstreamer-devel.966125.n4.nabble.com/
v4l2src is native gstreamer plugin and gstreamer forum users may have experience in running it with realsense cameras.

Hi

I have a fps question. ./deepstream-test5-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8_color_display.txt and i set 1920x1080 30 fps in the conig file.

but the fps only show 11

===============================================================
** INFO: <bus_callback:175>: Pipeline running

NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 0

**PERF: FPS 0 (Avg)
**PERF: 11.01 (11.01)
**PERF: 11.16 (11.09)
**PERF: 11.04 (11.07)

below is v4l2 dump it shows 1920x1080 can do 30fps… anything wrong ?

v4l2-ctl -d /dev/video2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘YUYV’
Name : YUYV 4:2:2
Size: Discrete 320x180
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 424x240
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x360
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 848x480
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 960x540
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: ''
Name        : 36315752-1a66-a242-9065-d01814a
	Size: Discrete 1920x1080
		Interval: Discrete 0.033s (30.000 fps)