i tried . similar err . with this config file source1_usb_dec_infer_resnet_int8.txt it can not read the engine file defined model-engine-file=…/…/models/Primary_Detector/resnet10.caffemodel_b30_int8.engine
(deepstream-app:16087): GStreamer-WARNING **: 17:24:58.750: Name ‘src_cap_filter’ is not unique in bin ‘src_sub_bin0’, not adding
*** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***
Opening in BLOCKING MODE
Creating LL OSD context new
0:00:02.900897010 16087 0x10f08d90 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:useEngineFile(): Failed to read from model engine file
0:00:02.900994147 16087 0x10f08d90 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:02.901259726 16087 0x10f08d90 WARN nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): INT8 not supported by platform. Trying FP16 mode.
0:02:09.345437539 16087 0x10f08d90 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /opt/nvidia/deepstream/deepstream-4.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_fp16.engine
Runtime commands:
h: Print this help
q: Quit
p: Pause
r: Resume
NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.
I got deepstream-test5-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8.txt
working. but I have to do below two things
Disable [primary-gie] becuase there is no resnet10 in part of deepstream sdk
enable=0
model-engine-file=…/…/models/Primary_Detector/resnet10.
2.I noticed that there are 3 dev node /dev/video0 /dev/video1/ /dev/video2
so in the config file source0 I tried camera-v4l2-dev-node=1 and camera-v4l2-dev-node=2 they both works . but only camera-v4l2-dev-node=0 is failing.
(deepstream-test5-app:25415): GStreamer-WARNING **: 23:05:22.440: Name ‘src_cap_filter’ is not unique in bin ‘src_sub_bin0’, not adding
*** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***
Opening in BLOCKING MODE
Creating LL OSD context new
Runtime commands:
h: Print this help
q: Quit
p: Pause
r: Resume
NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
To go back to the tiled display, right-click anywhere on the window.
** INFO: <bus_callback:189>: Pipeline ready
** INFO: <bus_callback:175>: Pipeline running
ERROR from src_elem: Internal data stream error.
Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstV4l2Src:src_elem:
streaming stopped, reason not-negotiated (-4)
Quitting
App run failed
=================================================================================
here is v4l2-ctl dump on /dev/video0
realsense camera provide depth image and rgb image i saw /dev/video2 is RGB image but i dont know how the depth image are create out of /dev/video0 and /dev/video1 . probably realsense sdk do some other processing to create the depth image. if that is the case. how do I plugin the realsense sdk driver into deepstream. should I write a application on top of realsense sdk . so instead deepstream read from /dev/video0 1 2 it read from my application as src into deepstream ?
Hi,
Please run ‘gst-inspect-1.0 v4l2src’ to get supported formats. It looks like the non-working formats are not supported by gstreamer frameworks. Since DS SDK is based on gstreamer, some device nodes on RealSense cameras may not be opened. A general usecase of USB cameras is to run in YUV422 such as UYVY, YUY2.
One possible solution is to add one type(ex: type=6) to support appsrc. In the implementation, you need to convert Z16 into NV12 and create source bin like
gst-inspect-1.0 v4l2src
Factory Details:
Rank primary (256)
Long-name Video (video4linux2) Source
Klass Source/Video
Description Reads frames from a Video4Linux2 device
Author Edgard Lima edgard.lima@gmail.com, Stefan Kost ensonic@users.sf.net
Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so
Version 1.14.5
License LGPL
Source module gst-plugins-good
Source release date 2019-05-29
Binary package GStreamer Good Plugins (Ubuntu)
Origin URL gst-plugins-good1.0 package : Ubuntu
Hi,
I think gstreamer does not support z16 in v4l2src plugin. But maybe I am wrong since I don’t have experience to sun realsense camera in gstreamer pipeline.
Could you please go to http://gstreamer-devel.966125.n4.nabble.com/
v4l2src is native gstreamer plugin and gstreamer forum users may have experience in running it with realsense cameras.
I have a fps question. ./deepstream-test5-app -c configs/deepstream-app/source1_usb_dec_infer_resnet_int8_color_display.txt and i set 1920x1080 30 fps in the conig file.
Hi,
Please run ‘sudo jetson_clocks’ to fix CPU cores at max clock. Also please try following setting:
# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=../../models/Primary_Detector_Nano/resnet10.caffemodel_b1_fp16.engine
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=4
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_nano.txt
The default [primary-gie] is for Xavier. Above setting is from source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt
ad shall be suitable for Jetson Nano.
Actually I will add inference later but not now. Let’s why I disable the inference in deepstream config and only check the fps and found fps is low comparing reading directly from realsense lib and send over rtps wiht gstreamer.
I expect deepstream has same performance reading from v4l2 vs reading frame from realsense lib on cpu. but looks like v4l2 path is slow. sudo jetson_clocks does not seem help on fps
Hi,
We can reach 720p30 on Jetson Nano + Realsense430.
Device information:
[ 107.847424] usb 2-1.1: new SuperSpeed USB device number 3 using tegra-xusb
[ 107.871179] usb 2-1.1: New USB device found, idVendor=8086, idProduct=0ad4
[ 107.871245] usb 2-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 107.871296] usb 2-1.1: Product: Intel(R) RealSense(TM) Depth Camera 430
[ 107.871339] usb 2-1.1: Manufacturer: Intel(R) RealSense(TM) Depth Camera 430
[ 107.871379] usb 2-1.1: SerialNumber: 824613020664
Config file:
# Copyright (c) 2018 NVIDIA Corporation. All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto. Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.
[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl
[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP 5=CSI
type=1
camera-width=1280
camera-height=720
camera-fps-n=30
camera-fps-d=1
camera-v4l2-dev-node=1
[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=5
sync=0
display-id=0
offset-x=0
offset-y=0
width=0
height=0
overlay-id=1
source-id=0
[osd]
enable=1
border-width=2
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
[streammux]
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1280
height=720
# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
model-engine-file=../../models/Primary_Detector_Nano/resnet10.caffemodel_b1_fp16.engine
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=4
#Required by the app for SGIE, when used along with config-file property
gie-unique-id=1
config-file=config_infer_primary_nano.txt
[tests]
file-loop=0
720p30 is the maximum resolution of Realsense 430 in UYVY format. You can modify it to 1080p30 and try realsense 415.
I set sudo jetson_clocks and used your config for 720p. i dont have resnet10.caffemodel_b1_fp16.engine in the deepstream directory. so i used resnet10.caffemodel
deepstream-app -c configs/deepstream-app/myconfig.txt
when use camera-v4l2-dev-node=1 i can get 30fps but when i used camera-v4l2-dev-node=2 (this is rgb data ) i only got 22.62 fps. can you try camera-v4l2-dev-node=2 ? attached tegrastat dump
Hi,
You should run ‘sudo tegrastats’ to get %.
resnet10.caffemodel_b1_fp16.engine is auto generated. It should appear after first-run.
Suggest you follow #23 to check if the source can reach 720p30. If the source is 30fps but only 22fps in running deepstream-app, it probably hits limitation of Jetson Nano. You may try larger interval setting such as interval=6(or 8).
We don’t see /dev/video2 when connecting D430. It should be a different case comparing to D415.