Jetson nano rtsp stream failed with intel realsense 415 usb camera

i tried . similar err . with this config file source1_usb_dec_infer_resnet_int8.txt it can not read the engine file defined model-engine-file=…/…/models/Primary_Detector/resnet10.caffemodel_b30_int8.engine

deepstream-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt

(deepstream-app:16087): GStreamer-WARNING **: 17:24:58.750: Name ‘src_cap_filter’ is not unique in bin ‘src_sub_bin0’, not adding

*** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***

Opening in BLOCKING MODE
Creating LL OSD context new
0:00:02.900897010 16087 0x10f08d90 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:useEngineFile(): Failed to read from model engine file
0:00:02.900994147 16087 0x10f08d90 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:02.901259726 16087 0x10f08d90 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): INT8 not supported by platform. Trying FP16 mode.
0:02:09.345437539 16087 0x10f08d90 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_fp16.engine

Runtime commands:
h: Print this help
q: Quit

p: Pause
r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.

**PERF: FPS 0 (Avg)
**PERF: 0.00 (0.00)
** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

ERROR from src_elem: Internal data stream error.
Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming stopped, reason not-negotiated (-4)
Quitting
App run failed

I got deepstream-test5-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt
working. but I have to do below two things

  1. Disable [primary-gie] becuase there is no resnet10 in part of deepstream sdk
    enable=0
    model-engine-file=…/…/models/Primary_Detector/resnet10.

2.I noticed that there are 3 dev node /dev/video0 /dev/video1/ /dev/video2
so in the config file source0 I tried camera-v4l2-dev-node=1 and camera-v4l2-dev-node=2 they both works . but only camera-v4l2-dev-node=0 is failing.

21 [source0]
22 enable=1
23 #Type - 1=CameraV4L2 2=URI 3=MultiURI
24 type=1
25 camera-width=1280
26 camera-height=720
27 camera-fps-n=30
28 camera-fps-d=1
29 camera-v4l2-dev-node=1 //Terry 2 also works

=================================================================================
deepstream-test5-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt

(deepstream-test5-app:25415): GStreamer-WARNING **: 23:05:22.440: Name ‘src_cap_filter’ is not unique in bin ‘src_sub_bin0’, not adding

*** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***

Opening in BLOCKING MODE
Creating LL OSD context new

Runtime commands:
h: Print this help
q: Quit

p: Pause
r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.

** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

ERROR from src_elem: Internal data stream error.
Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming stopped, reason not-negotiated (-4)
Quitting
App run failed

=================================================================================
here is v4l2-ctl dump on /dev/video0

v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'Z16 ’
Name : Depth data 16-bit (Z16)
Size: Discrete 256x144
Interval: Discrete 0.011s (90.000 fps)
Size: Discrete 424x240
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 480x270
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x360
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 848x100
Interval: Discrete 0.010s (100.000 fps)
Size: Discrete 848x480
Interval: Discrete 0.011s (90.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: ''
Name        : 00000050-0000-0010-8000-00aa003
	Size: Discrete 256x144
		Interval: Discrete 0.011s (90.000 fps)
	Size: Discrete 424x240
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 480x270
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 640x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 848x100
		Interval: Discrete 0.010s (100.000 fps)
	Size: Discrete 848x480
		Interval: Discrete 0.011s (90.000 fps)
		Interval: Discrete 0.017s (60.000 fps)
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.167s (6.000 fps)
  1. realsense camera provide depth image and rgb image i saw /dev/video2 is RGB image but i dont know how the depth image are create out of /dev/video0 and /dev/video1 . probably realsense sdk do some other processing to create the depth image. if that is the case. how do I plugin the realsense sdk driver into deepstream. should I write a application on top of realsense sdk . so instead deepstream read from /dev/video0 1 2 it read from my application as src into deepstream ?

Thanks,
Terry

Hi,
Please run ‘gst-inspect-1.0 v4l2src’ to get supported formats. It looks like the non-working formats are not supported by gstreamer frameworks. Since DS SDK is based on gstreamer, some device nodes on RealSense cameras may not be opened. A general usecase of USB cameras is to run in YUV422 such as UYVY, YUY2.

sure i will run it tomorrow . but could you tell me where to change the format in the configuration file ?

also if node is not opened as you suggested . Can I read the depth data from realsense application and send it to Deepstream as input ?

Hi,
By default we have the implementations in source group:
https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html#page/DeepStream_Development_Guide%2Fdeepstream_app_config.3.2.html%23wwpID0E0QB0HA

One possible solution is to add one type(ex: type=6) to support appsrc. In the implementation, you need to convert Z16 into NV12 and create source bin like

appsrc ! video/x-raw,format=NV12 ! nvvideoconvert ! 'video/x-raw(memory:NMM),format=NV12'

deepstream-app is open source and you can customize it into the usecase.
Other users may have other possible solutions to share here.

gst-inspect-1.0 v4l2src
Factory Details:
Rank primary (256)
Long-name Video (video4linux2) Source
Klass Source/Video
Description Reads frames from a Video4Linux2 device
Author Edgard Lima edgard.lima@gmail.com, Stefan Kost ensonic@users.sf.net

Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so
Version 1.14.5
License LGPL
Source module gst-plugins-good
Source release date 2019-05-29
Binary package GStreamer Good Plugins (Ubuntu)
Origin URL gst-plugins-good1.0 package : Ubuntu

GObject
±—GInitiallyUnowned
±—GstObject
±—GstElement
±—GstBaseSrc
±—GstPushSrc
±—GstV4l2Src

Implemented Interfaces:
GstURIHandler
GstTuner
GstColorBalance
GstVideoOrientation

Pad Templates:
SRC template: ‘src’
Availability: Always
Capabilities:
image/jpeg
video/mpeg
mpegversion: 4
systemstream: false
video/mpeg
mpegversion: 2
video/mpegts
systemstream: true
video/x-bayer
format: { (string)bggr, (string)gbrg, (string)grbg, (string)rggb }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-dv
systemstream: true
video/x-h263
variant: itu
video/x-h264
stream-format: { (string)byte-stream, (string)avc }
alignment: au
video/x-pwc1
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-pwc2
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw
format: { (string)RGB16, (string)BGR, (string)RGB, (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)YVU9, (string)YV12, (string)YUY2, (string)YVYU, (string)UYVY, (string)Y42B, (string)Y41B, (string)YUV9, (string)NV12_64Z32, (string)NV24, (string)NV61, (string)NV16, (string)NV21, (string)NV12, (string)I420, (string)BGRA, (string)BGRx, (string)ARGB, (string)xRGB, (string)BGR15, (string)RGB15 }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-sonix
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-vp8
video/x-vp9
video/x-wmv
wmvversion: 3
format: WVC1

Element has no clocking capabilities.

URI handling capabilities:
Element can act as source.
Supported URI protocols:
v4l2

Pads:
SRC: ‘src’
Pad Template: ‘src’

Element Properties:
name : The name of the object
flags: readable, writable
String. Default: “v4l2src0”
parent : The parent of the object
flags: readable, writable
Object of type “GstObject”
blocksize : Size in bytes to read per buffer (-1 = default)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 4096
num-buffers : Number of buffers to output before sending EOS (-1 = unlimited)
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
typefind : Run typefind before negotiating (deprecated, non-functional)
flags: readable, writable, deprecated
Boolean. Default: false
do-timestamp : Apply current stream time to buffers
flags: readable, writable
Boolean. Default: false
device : Device location
flags: readable, writable
String. Default: “/dev/video0”
device-name : Name of the device
flags: readable
String. Default: null
device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1
flags : Device type flags
flags: readable
Flags “GstV4l2DeviceTypeFlags” Default: 0x00000000, “(none)”
(0x00000001): capture - Device supports video capture
(0x00000002): output - Device supports video playback
(0x00000004): overlay - Device supports video overlay
(0x00000010): vbi-capture - Device supports the VBI capture
(0x00000020): vbi-output - Device supports the VBI output
(0x00010000): tuner - Device has a tuner or modulator
(0x00020000): audio - Device has audio inputs or outputs
brightness : Picture brightness, or more precisely, the black level
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
contrast : Picture contrast or luma gain
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
saturation : Picture color saturation or chroma gain
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
hue : Hue or color balance
flags: readable, writable, controllable
Integer. Range: -2147483648 - 2147483647 Default: 0
norm : video standard
flags: readable, writable
Enum “V4L2_TV_norms” Default: 0, “none”
(0): none - none
(45056): NTSC - NTSC
(4096): NTSC-M - NTSC-M
(8192): NTSC-M-JP - NTSC-M-JP
(32768): NTSC-M-KR - NTSC-M-KR
(16384): NTSC-443 - NTSC-443
(255): PAL - PAL
(7): PAL-BG - PAL-BG
(1): PAL-B - PAL-B
(2): PAL-B1 - PAL-B1
(4): PAL-G - PAL-G
(8): PAL-H - PAL-H
(16): PAL-I - PAL-I
(224): PAL-DK - PAL-DK
(32): PAL-D - PAL-D
(64): PAL-D1 - PAL-D1
(128): PAL-K - PAL-K
(256): PAL-M - PAL-M
(512): PAL-N - PAL-N
(1024): PAL-Nc - PAL-Nc
(2048): PAL-60 - PAL-60
(16711680): SECAM - SECAM
(65536): SECAM-B - SECAM-B
(262144): SECAM-G - SECAM-G
(524288): SECAM-H - SECAM-H
(3276800): SECAM-DK - SECAM-DK
(131072): SECAM-D - SECAM-D
(1048576): SECAM-K - SECAM-K
(2097152): SECAM-K1 - SECAM-K1
(4194304): SECAM-L - SECAM-L
(8388608): SECAM-Lc - SECAM-Lc
io-mode : I/O mode
flags: readable, writable
Enum “GstV4l2IOMode” Default: 0, “auto”
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
extra-controls : Extra v4l2 controls (CIDs) for the device
flags: readable, writable
Boxed pointer of type “GstStructure”
pixel-aspect-ratio : Overwrite the pixel aspect ratio of the device
flags: readable, writable
String. Default: null
force-aspect-ratio : When enabled, the pixel aspect ratio will be enforced
flags: readable, writable
Boolean. Default: true

Element Signals:
“prepare-format” : void user_function (GstElement* object,
gint arg0,
GstCaps* arg1,
gpointer user_data);

Hi,
Don’t see Z16 is listed in

format: { (string)RGB16, (string)BGR, (string)RGB, (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, (string)YVU9, (string)YV12, (string)YUY2, (string)YVYU, (string)UYVY, (string)Y42B, (string)Y41B, (string)YUV9, (string)NV12_64Z32, (string)NV24, (string)NV61, (string)NV16, (string)NV21, (string)NV12, (string)I420, (string)BGRA, (string)BGRx, (string)ARGB, (string)xRGB, (string)BGR15, (string)RGB15 }

Please refer to #15 on customizing deepstream-app.

Hi

Could you please explain a bit more.

Is it because realsense does not support z16 format or streamer does not support z16 format?

What it takes to add z16 format support?

Thanks

Hi,
I think gstreamer does not support z16 in v4l2src plugin. But maybe I am wrong since I don’t have experience to sun realsense camera in gstreamer pipeline.

Could you please go to
http://gstreamer-devel.966125.n4.nabble.com/
v4l2src is native gstreamer plugin and gstreamer forum users may have experience in running it with realsense cameras.

Hi

I have a fps question. ./deepstream-test5-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8_color_display.txt and i set 1920x1080 30 fps in the conig file.

but the fps only show 11

===============================================================
** INFO: <bus_callback:175>: Pipeline running

NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 0

**PERF: FPS 0 (Avg)
**PERF: 11.01 (11.01)
**PERF: 11.16 (11.09)
**PERF: 11.04 (11.07)

below is v4l2 dump it shows 1920x1080 can do 30fps… anything wrong ?

v4l2-ctl -d /dev/video2 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘YUYV’
Name : YUYV 4:2:2
Size: Discrete 320x180
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 424x240
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x360
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 848x480
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 960x540
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.167s (6.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: ''
Name        : 36315752-1a66-a242-9065-d01814a
	Size: Discrete 1920x1080
		Interval: Discrete 0.033s (30.000 fps)

Hi,
Please run ‘sudo jetson_clocks’ to fix CPU cores at max clock. Also please try following setting:

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=../../models/Primary_Detector_Nano/resnet10.caffemodel_b1_fp16.engine
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=4
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_nano.txt

The default [primary-gie] is for Xavier. Above setting is from
source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt
ad shall be suitable for Jetson Nano.

Thanks i will try the clock change btw since I dont need run inference . do I have to turn on [primary-gie] ?

also is there any tool to figure out why fps is low ?

Hi,
If you don’t need inference, it can be easier to run a gst-launch pipeline like:
https://devtalk.nvidia.com/default/topic/1057681/jetson-tx1/logitech-c930e-on-jetson-tx1-very-slow-and-choppy-video/post/5363417/#5363417
You can use fpsdisplaysink to check framerate:

$ gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw,format=YUY2,width=848,height=480,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! fpsdisplaysink text-overlay=0 video-sink=nvoverlaysink -v

Hi Dane,

Okay I will check gst-launch fps as well.

Actually I will add inference later but not now. Let’s why I disable the inference in deepstream config and only check the fps and found fps is low comparing reading directly from realsense lib and send over rtps wiht gstreamer.

I expect deepstream has same performance reading from v4l2 vs reading frame from realsense lib on cpu. but looks like v4l2 path is slow. sudo jetson_clocks does not seem help on fps

Hi,
We have verified four USB camera sources(two USB3 and two USB2 cameras) on Jetson Nano. Each can reach 640x480p30. The detail is in
https://devtalk.nvidia.com/default/topic/1067850/deepstream-sdk/connected-more-than-two-usb-cameras-problem-on-deepstream-app-jetson-nano-dev-kit-/post/5410823/#5410823

You can run tegrastats to check the system loading in running realsense 415.

Hi,
We can reach 720p30 on Jetson Nano + Realsense430.

Device information:

[  107.847424] usb 2-1.1: new SuperSpeed USB device number 3 using tegra-xusb
[  107.871179] usb 2-1.1: New USB device found, idVendor=8086, idProduct=0ad4
[  107.871245] usb 2-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  107.871296] usb 2-1.1: Product: Intel(R) RealSense(TM) Depth Camera 430
[  107.871339] usb 2-1.1: Manufacturer: Intel(R) RealSense(TM) Depth Camera 430
[  107.871379] usb 2-1.1: SerialNumber: 824613020664

Config file:

# Copyright (c) 2018 NVIDIA Corporation.  All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP 5=CSI
type=1
camera-width=1280
camera-height=720
camera-fps-n=30
camera-fps-d=1
camera-v4l2-dev-node=1

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=5
sync=0
display-id=0
offset-x=0
offset-y=0
width=0
height=0
overlay-id=1
source-id=0

[osd]
enable=1
border-width=2
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0

[streammux]
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1280
height=720

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
model-engine-file=../../models/Primary_Detector_Nano/resnet10.caffemodel_b1_fp16.engine
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=4
#Required by the app for SGIE, when used along with config-file property
gie-unique-id=1
config-file=config_infer_primary_nano.txt

[tests]
file-loop=0

720p30 is the maximum resolution of Realsense 430 in UYVY format. You can modify it to 1080p30 and try realsense 415.

Hi Dane,

I set sudo jetson_clocks and used your config for 720p. i dont have resnet10.caffemodel_b1_fp16.engine in the deepstream directory. so i used resnet10.caffemodel
deepstream-app -c configs/deepstream-app/myconfig.txt

when use camera-v4l2-dev-node=1 i can get 30fps but when i used camera-v4l2-dev-node=2 (this is rgb data ) i only got 22.62 fps. can you try camera-v4l2-dev-node=2 ? attached tegrastat dump

===================================
EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26C iwlwifi@28C PMIC@100C GPU@24.5C AO@30C thermal@25C POM_5V_IN 3230/3276 POM_5V_GPU 121/120 POM_5V_CPU 969/974
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [23%@1479,35%@1479,38%@1479,2%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26.5C iwlwifi@33C PMIC@100C GPU@24.5C AO@30C thermal@25C POM_5V_IN 3149/3270 POM_5V_GPU 121/120 POM_5V_CPU 888/970
RAM 1740/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [12%@1479,32%@1479,49%@1479,6%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26.5C iwlwifi@33C PMIC@100C GPU@24.5C AO@30C thermal@25.5C POM_5V_IN 3265/3270 POM_5V_GPU 120/120 POM_5V_CPU 967/970
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [23%@1479,42%@1479,49%@1479,29%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@27C iwlwifi@29C PMIC@100C GPU@24.5C AO@30C thermal@25.25C POM_5V_IN 3432/3277 POM_5V_GPU 120/120 POM_5V_CPU 967/970
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [20%@1479,33%@1479,46%@1479,3%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26C iwlwifi@28C PMIC@100C GPU@24.5C AO@30C thermal@25.25C POM_5V_IN 3230/3275 POM_5V_GPU 121/120 POM_5V_CPU 969/970
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [13%@1479,37%@1479,49%@1479,3%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@27C iwlwifi@28C PMIC@100C GPU@24.5C AO@30C thermal@25.5C POM_5V_IN 3351/3278 POM_5V_GPU 121/120 POM_5V_CPU 1048/973
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [26%@1479,35%@1479,40%@1479,2%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26C iwlwifi@28C PMIC@100C GPU@24.5C AO@30C thermal@25.75C POM_5V_IN 3154/3273 POM_5V_GPU 121/120 POM_5V_CPU 888/970
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [35%@1479,33%@1479,28%@1479,2%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26C iwlwifi@28C PMIC@100C GPU@24C AO@30.5C thermal@25.25C POM_5V_IN 3230/3271 POM_5V_GPU 121/120 POM_5V_CPU 969/970
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [29%@1479,18%@1479,46%@1479,6%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@27C iwlwifi@28C PMIC@100C GPU@24.5C AO@30C thermal@25.75C POM_5V_IN 3311/3273 POM_5V_GPU 121/120 POM_5V_CPU 1009/971
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [22%@1479,27%@1479,46%@1479,3%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@27C iwlwifi@28C PMIC@100C GPU@24.5C AO@30.5C thermal@25.75C POM_5V_IN 3230/3271 POM_5V_GPU 121/120 POM_5V_CPU 969/971
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [38%@1479,17%@1479,39%@1479,2%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26C iwlwifi@28C PMIC@100C GPU@24.5C AO@30.5C thermal@25.25C POM_5V_IN 3271/3271 POM_5V_GPU 121/120 POM_5V_CPU 969/971
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [12%@1479,35%@1479,52%@1479,2%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@27.5C iwlwifi@28C PMIC@100C GPU@24C AO@30C thermal@25.25C POM_5V_IN 3311/3273 POM_5V_GPU 121/120 POM_5V_CPU 969/971
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [24%@1479,30%@1479,46%@1479,1%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26.5C iwlwifi@28C PMIC@100C GPU@24.5C AO@30C thermal@25.5C POM_5V_IN 3311/3274 POM_5V_GPU 121/120 POM_5V_CPU 1009/972
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [35%@1479,32%@1479,27%@1479,1%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26C iwlwifi@28C PMIC@100C GPU@24C AO@30C thermal@25.25C POM_5V_IN 3392/3277 POM_5V_GPU 120/120 POM_5V_CPU 967/972
RAM 1741/3956MB (lfb 323x4MB) SWAP 0/1978MB (cached 0MB) CPU [17%@1479,37%@1479,48%@1479,1%@1479] EMC_FREQ 0% GR3D_FREQ 0% PLL@22C CPU@26C iwlwifi@28C PMIC@100C GPU@24.5C AO@30C thermal@25.25C POM_5V_IN 3230/3276 POM_5V_GPU 121/120 POM_5V_CPU 969/972

===================================

attached also the perf print out when input is 720p ncamera-v4l2-dev-node=2

**PERF: FPS 0 (Avg)
**PERF: 22.70 (22.64)
**PERF: 22.65 (22.64)
**PERF: 22.37 (22.64)
**PERF: 22.55 (22.64)
**PERF: 22.92 (22.64)
**PERF: 22.42 (22.64)
**PERF: 22.35 (22.64)
**PERF: 22.77 (22.64)
**PERF: 22.75 (22.64)
**PERF: 22.41 (22.64)
**PERF: 22.75 (22.64)
**PERF: 22.60 (22.64)
**PERF: 22.72 (22.64)
**PERF: 22.57 (22.64)
**PERF: 22.94 (22.64)
**PERF: 22.52 (22.64)
**PERF: 22.26 (22.64)

Hi,
You should run ‘sudo tegrastats’ to get %.
resnet10.caffemodel_b1_fp16.engine is auto generated. It should appear after first-run.

Suggest you follow #23 to check if the source can reach 720p30. If the source is 30fps but only 22fps in running deepstream-app, it probably hits limitation of Jetson Nano. You may try larger interval setting such as interval=6(or 8).

We don’t see /dev/video2 when connecting D430. It should be a different case comparing to D415.