deepstream4.0 deepstream-app cannot show the frame by rtsp

Hi, I use the demo of deepstream_sdk_v4.0_x86_64/sources/apps/sample_apps/deepstream-app.

It works when using the command of “deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt” to read from video file
.
the source0 config is that:
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=2
uri=file://…/…/streams/sample_1080p_h264.mp4
num-sources=1
#drop-frame-interval=2
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

But it donot show the frame when read from rtsp.
The source0 config is that:
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=2
uri=rtsp://root:root@127.0.0.1:8554/session0.mpg
num-sources=1
#drop-frame-interval=2
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

We get the YV12 pixel data from video capture card and use h264 to compress the pixel data。
Then I use the software to send the video data stream by rtsp.
The all frames of the demo deepstream-app showing is wrong, it is all gray color。 But when I use the vlc to receive the video data stream by rtsp, it can show the right frame.
I wannt to know why the demo of deepstream-app cannot show the right frame by rtsp. However the vlc can show the right frame from the same rtsp source.

Hi,
Please try type=4

Hi, I use type=4, the demo of deepstream-app also cannot show the right frame by rtsp, it is also all gray color.
This problem has trouble me few days.

The source0 config is that:
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://root:root@127.0.0.1:8554/session0.mpg
num-sources=1
#drop-frame-interval=2
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

I get only the document of DeepStream_Plugin_Manual.pdf,it cannot enough to help me to find the problem. Is there any more document that can help me?

thank you very much.

the complete configure is:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=0
rows=2
columns=2
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
#uri=file://…/…/streams/sample_1080p_h264.mp4
uri=rtsp://root:root@127.0.0.1:8554/session0.mpg
num-sources=1
#drop-frame-interval=2
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=1
source-id=0
gpu-id=0
nvbuf-memory-type=0

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
sync=0
#iframeinterval=10
bitrate=2000000
output-file=out.mp4
source-id=0

[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=4
#1=h264 2=h265
codec=1
sync=0
#iframeinterval=10
bitrate=400000

set below properties in case of RTSPStreaming

rtsp-port=8554
udp-port=5400

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=4000000

Set muxer output width and height

width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

[primary-gie]
enable=1
gpu-id=0
#model-engine-file=…/…/models/Primary_Detector/resnet10.caffemodel_b4_int8.engine
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt

[tracker]
enable=1
tracker-width=1920
tracker-height=1080
#ll-lib-file=/opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_mot_iou.so
#ll-lib-file=/opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_nvdcf.so
ll-lib-file=/opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_mot_klt.so
#ll-config-file required for DCF/IOU only
#ll-config-file=tracker_config.yml
#ll-config-file=iou_config.txt
gpu-id=0
#enable-batch-process applicable to DCF only
enable-batch-process=1

[secondary-gie0]
enable=1
model-engine-file=…/…/models/Secondary_VehicleTypes/resnet18.caffemodel_b16_int8.engine
gpu-id=0
batch-size=1
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_vehicletypes.txt

[secondary-gie1]
enable=1
model-engine-file=…/…/models/Secondary_CarColor/resnet18.caffemodel_b16_int8.engine
batch-size=1
gpu-id=0
gie-unique-id=5
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_carcolor.txt

[secondary-gie2]
enable=1
model-engine-file=…/…/models/Secondary_CarMake/resnet18.caffemodel_b16_int8.engine
batch-size=1
gpu-id=0
gie-unique-id=6
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_carmake.txt

[tests]
file-loop=0

And When I use the demo of deepstream-app to receive data by rtsp and save mp4 file.

The configure is:
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
#uri=file://…/…/streams/sample_1080p_h264.mp4
uri=rtsp://root:root@127.0.0.1:8554/session0.mpg
#uri=rtsp://127.0.0.1:8554/
num-sources=1
#drop-frame-interval=2
gpu-id=0

(0): memtype_device - Memory type Device

(1): memtype_pinned - Memory type Host Pinned

(2): memtype_unified - Memory type Unified

cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
sync=0
#iframeinterval=10
bitrate=2000000
output-file=out.mp4

the debug info of deepstream-app is that:

Runtime commands:
h: Print this help
q: Quit

    p: Pause
    r: Resume

** INFO: <bus_callback:165>: Pipeline ready

** INFO: <bus_callback:151>: Pipeline running

Creating LL OSD context new
KLT Tracker Init
ERROR from sink_sub_bin_mux1: Could not multiplex stream.
Debug info: gstqtmux.c(4561): gst_qt_mux_add_buffer (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/
GstQTMux:sink_sub_bin_mux1:
Buffer has no PTS
.
Quitting
ERROR from sink_sub_bin_queue1: Internal data stream error.
Debug info: gstqueue.c(1566): gst_queue_loop (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/GstQueue:sink_sub_bin_queue1:
streaming stopped, reason error (-5)
App run failed

Hi,
Please share your machine. Is it a Jetson platform or PC with NVIDIA GPUs?

Hi, It is PC with NVIDIA GPUs of GTX1060.
And t works when using the command of “deepstream-app -c source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt” to read from video file.
The proble is that it does not show the right frame by rtsp
, it is also all gray color.
Thank you.

Hi,
Please run below commands to check if you can play the source and dump into good jpeg files. And if nvv4l2decoder is picked by uridecodebin.

$ export GST_DEBUG=*FACTORY*:4
$ gst-launch-1.0 uridecodebin uri="rtsp://root:root@127.0.0.1:8554/session0.mpg" ! nvvideoconvert ! jpegenc ! multifilesink location=dump%05d.jpg

Hi,
I try to test with your commands.
I play the source. It dumps into bad jpeg files.
I can open the jpeg files.
The first picture has other colors, but it is dim, so it is not right.
Other pictures are all gray color.

thank you.

the debug info is:
gst-launch-1.0 uridecodebin uri=“rtsp://root:root@127.0.0.1:8554/session0.mpg” ! nvvideoconvert ! jpegenc ! multifilesink location=dump%05d.jpg
0:00:00.053180697 20239 0x560925d12920 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “uridecodebin”
0:00:00.078528711 20239 0x560925d12920 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nvvideoconvert”
0:00:00.079363942 20239 0x560925d12920 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “jpegenc”
0:00:00.081575618 20239 0x560925d12920 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “multifilesink”
0:00:00.081796569 20239 0x560925d12920 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “pipeline”
Setting pipeline to PAUSED …
0:00:00.083936259 20239 0x560925d12920 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “rtspsrc” named “source”
0:00:00.084939821 20239 0x560925d12920 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtspwms”
0:00:00.085378092 20239 0x560925d12920 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtspreal”
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://root:root@127.0.0.1:8554/session0.mpg
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
0:00:00.095220123 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsrc”
0:00:00.095794403 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsrc”
Progress: (request) SETUP stream 0
0:00:00.097862419 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “rtpbin” named “manager”
0:00:00.098383545 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpsession”
0:00:00.098641205 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpssrcdemux”
0:00:00.099432889 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpstorage”
0:00:00.099955333 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “funnel”
0:00:00.100012596 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “funnel”
0:00:00.102629510 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsink”
0:00:00.103615327 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “fakesrc”
0:00:00.103884680 20239 0x560925dbd280 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsink”
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:00.105399228 20239 0x7f0aac05b1e0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpjitterbuffer”
0:00:00.105544551 20239 0x7f0aac05b1e0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpptdemux”
0:00:00.171043671 20239 0x7f0a7c004720 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “decodebin”
0:00:00.171351127 20239 0x7f0a7c004720 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “typefind” named “typefind”
0:00:00.178265788 20239 0x7f0a7c004720 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtph264depay”
0:00:00.180670350 20239 0x7f0a7c004720 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “h264parse”
0:00:00.181503060 20239 0x7f0a7c004720 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “capsfilter”
0:00:00.247810287 20239 0x7f0a7c004720 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nvv4l2decoder”
^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:07.297105403
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

I also use vlc to receive the same video data by rtsp.
the vlc can show the right frame from the same rtsp

Hi,
Could you try deepstream-test3? It also uses uridecodebin.
Besides, is your source in standard resolution such as 1280x720 or 1920x1080?

Hi,
I try deepstream-test3 to receive the same rtsp, it is also wrong.
It is also all gray color.
The source is 1920*1080.

the deepstream-test3 debug info is:
Now playing: rtsp://root:root@127.0.0.1:8554/session0.mpg,
Creating LL OSD context new
0:00:00.835852500 29908 0x5555562ad470 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger: NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:10.323329611 29908 0x5555562ad470 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger: NvDsInferContext[UID 1]:generateTRTModel(): Storing the serialized cuda engine to file at /home/linux/tool/deepstream4.0/deepstream/deepstream_sdk_v4.0_x86_64/samples/models/Primary_Detector/resnet10.caffemodel_b1_int8.engine
Decodebin child added: source
Running…
Decodebin child added: decodebin0
Decodebin child added: rtph264depay0
Decodebin child added: h264parse0
Decodebin child added: capsfilter0
Decodebin child added: nvv4l2decoder0
Seting bufapi_version

(deepstream-test3-app:29908): GLib-GObject-WARNING **: 19:20:44.080: g_object_set_is_valid_property: object class ‘nvv4l2decoder’ has no property named ‘bufapi-version’
In cb_newpad
Creating LL OSD context new
Frame Number = 0 Number of objects = 0 Vehicle Count = 0 Person Count = 0
Frame Number = 1 Number of objects = 0 Vehicle Count = 0 Person Count = 0
Frame Number = 2 Number of objects = 0 Vehicle Count = 0 Person Count = 0
Frame Number = 3 Number of objects = 0 Vehicle Count = 0 Person Count = 0
Frame Number = 4 Number of objects = 0 Vehicle Count = 0 Person Count = 0
Frame Number = 5 Number of objects = 0 Vehicle Count = 0 Person Count = 0
Frame Number = 6 Number of objects = 0 Vehicle Count = 0 Person Count = 0

Hi,
If your source is not 1920x1080, please modify the default setting and try again.

#define MUXER_OUTPUT_WIDTH 1920
#define MUXER_OUTPUT_HEIGHT 1080
#define TILED_OUTPUT_WIDTH 1920
#define TILED_OUTPUT_HEIGHT 1080