RTSP Streaming on DeepStream 5.0 Jetson Nano Pixel distorted

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I am running multi-camera RTSP(h264 encoding) but pixels starts distorting when there is motion in camera or sometimes frame pixels distort or break a lot.

My config file:

cat source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt

[application]

enable-perf-measurement=1

perf-measurement-interval-sec=5

#gie-kitti-output-dir=streamscl

[tiled-display]

enable=1

rows=2

columns=2

width=1280

height=720

gpu-id=0

#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform

#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla

#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla

#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla

#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson

nvbuf-memory-type=0

[source0]

enable=1

#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP

type=4

#uri=file://…/…/streams/sample_1080p_h264.mp4

#uri=rtsp://facit:FacitData123@192.168.0.103:554/profile2/media.smp

uri=rtsp://facit:FacitData123@192.168.0.103:554/Test/media.smp

#num-sources=1

#drop-frame-interval=2

gpu-id=0

#(0): memtype_device - Memory type Device

#(1): memtype_pinned - Memory type Host Pinned

#(2): memtype_unified - Memory type Unified

cudadec-memtype=0

camera-fps-n=25

camera-fps-d=1

#latency=0

[source1]

enable=1

#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP

type=4

#uri=file://…/…/streams/sample_1080p_h264.mp4

uri=rtsp://facit:FacitData123@192.168.0.104:554/Test/media.smp

#num-sources=1

#drop-frame-interval=2

gpu-id=0

#(0): memtype_device - Memory type Device

#(1): memtype_pinned - Memory type Host Pinned

#(2): memtype_unified - Memory type Unified

cudadec-memtype=0

camera-fps-n=25

camera-fps-d=1

#latency=200

[source2]

enable=1

#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP

type=4

#uri=file://…/…/streams/sample_1080p_h264.mp4

uri=rtsp://facit:FacitData123@192.168.0.106:554/Test/media.smp

#num-sources=1

#drop-frame-interval=2

gpu-id=0

#(0): memtype_device - Memory type Device

#(1): memtype_pinned - Memory type Host Pinned

#(2): memtype_unified - Memory type Unified

cudadec-memtype=0

camera-fps-n=25

camera-fps-d=1

[source3]

enable=1

#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP

type=4

#uri=file://…/…/streams/sample_1080p_h264.mp4

uri=rtsp://facit:facit@192.168.0.91:554/axis-media/media.amp

#num-sources=1

#drop-frame-interval=2

gpu-id=0

#(0): memtype_device - Memory type Device

#(1): memtype_pinned - Memory type Host Pinned

#(2): memtype_unified - Memory type Unified

cudadec-memtype=0

camera-fps-n=25

camera-fps-d=1

[sink0]

enable=1

#Type - 1=FakeSink 2=EglSink 3=File

type=2

sync=0

source-id=0

gpu-id=0

qos=0

nvbuf-memory-type=0

#overlay-id=1

[sink1]

enable=0

type=3

#1=mp4 2=mkv

container=1

#1=h264 2=h265

codec=1

#encoder type 0=Hardware 1=Software

enc-type=0

sync=0

#iframeinterval=10

bitrate=2000000

#H264 Profile - 0=Baseline 2=Main 4=High

#H265 Profile - 0=Main 1=Main10

profile=0

output-file=out.mp4

source-id=0

[sink2]

enable=1

#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming

type=4

#1=h264 2=h265

codec=1

#encoder type 0=Hardware 1=Software

enc-type=0

sync=0

bitrate=4000000

#H264 Profile - 0=Baseline 2=Main 4=High

#H265 Profile - 0=Main 1=Main10

profile=0

set below properties in case of RTSPStreaming

rtsp-port=554

udp-port=5400

[osd]

enable=1

gpu-id=0

border-width=1

text-size=15

text-color=1;1;1;1;

text-bg-color=0.3;0.3;0.3;1

font=Serif

show-clock=0

clock-x-offset=800

clock-y-offset=820

clock-text-size=12

clock-color=1;0;0;0

nvbuf-memory-type=0

[streammux]

gpu-id=0

##Boolean property to inform muxer that sources are live

live-source=1

batch-size=6

##time out in usec, to wait after the first buffer is available

##to push the batch even if the complete batch is not formed

batched-push-timeout=40000

Set muxer output width and height

width=1280

height=720

##Enable to maintain aspect ratio wrt source, and allow black borders, works

##along with width, height properties

enable-padding=0

nvbuf-memory-type=0

If set to TRUE, system timestamp will be attached as ntp timestamp

If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached

attach-sys-ts-as-ntp=1

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

[primary-gie]

enable=0

gpu-id=0

model-engine-file=…/…/models/Primary_Detector_Nano/resnet10.caffemodel_b8_gpu0_fp16.engine

batch-size=1

#Required by the app for OSD, not a plugin property

bbox-border-color0=1;0;0;1

bbox-border-color1=0;1;1;1

bbox-border-color2=0;0;1;1

bbox-border-color3=0;1;0;1

interval=0

gie-unique-id=1

nvbuf-memory-type=0

config-file=config_infer_primary_nano.txt

[tracker]

enable=0

For the case of NvDCF tracker, tracker-width and tracker-height must be a multiple of 32, respectively

tracker-width=640

tracker-height=384

#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_iou.so

#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_nvdcf.so

ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so

#ll-config-file required for DCF/IOU only

#ll-config-file=tracker_config.yml

#ll-config-file=iou_config.txt

gpu-id=0

#enable-batch-process and enable-past-frame applicable to DCF only

enable-batch-process=1

enable-past-frame=0

display-tracking-id=0

[tests]

file-loop=0

console:

Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
Opening in BLOCKING MODE
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 279
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 279
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
H264: Profile = 66, Level = 0
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
** INFO: <bus_callback:167>: Pipeline running

Hardware Specs:
Hardware Platform: Jetson Nano
DeepStream Version: 5.0 latest version
JetPack Version: 4.4 latest [L4T 32.4.3]
TensorRT Version: 7.1.3.0
NVIDIA GPU Driver Version: CUDA 10.2

3

You have set up two output. one is on-screen output direct to panel, the other is rtsp stream sending out. Where did you observe the distortion? On the device’s panel or from the rtsp stream received in another device?

4

@Fiona.Chen
Yes. The distortion is on panel screen and even if I disable sink2 (i.e output streaming sending out via rtsp) still i get the distortion in on-screen output panel.

5

Can you try to disable osd and tiler in the configuration first for test the multiple rtsp streams themselves only? Since the nvinfer and tracker are already disabled in your configuration, so we need to make sure that it is not nvdsosd or nvmultistreamtiler distort the images.

If the streams still distorted after disable osd and tiler, it is probably network package lost during rtsp transmission caused the distortion.