Pixel distortion while streaming through RTSP cameras in DeepStream 5.1

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
#1

Hi Nvidia Team,

I am running the Yolov3 model e Inference on Jetson Nano using DeepStream SDK 5.1 with 4 Cameras(3 are RTSP and one is Webcam). But during the Inference, the pixels get distorted for RTSP Cameras. The image of the same have been attached below.

IMG_20210323_111431
IMG_20210323_1114314000×3000 1.11 MB

I am not sure why there is a Pixel Distortion for RTSP Cameras.

I am attaching the Config below:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1

[tiled-display]
enable=1
rows=2
columns=2
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=1
num-sources=1
gpu-id=0
camera-width=640
camera-height=480
camera-fps-n=20
camera-fps-d=1
camera-v4l2-dev-node=0
cudadec-memtype=0

[source1]
enable=1
type=4
uri=rtsp://[admin:sl123456@192.168.1.251:554/streaming/channels/102](http://admin:sl123456@192.168.1.251:554/streaming/channels/102)
num-sources=1
gpu-id=0
cudadec-memtype=0

[source2]
enable=1
type=4
uri=rtsp://[admin:admin123@192.168.1.252:554/cam/realmonitor?channel=1&subtype=1](http://admin:admin123@192.168.1.252:554/cam/realmonitor?channel=1&subtype=1)
num-sources=1
gpu-id=0
cudadec-memtype=0

[source3]
enable=1
type=4
uri=rtsp://[admin:admin123@192.168.1.253:554/cam/realmonitor?channel=1&subtype=1](http://admin:admin123@192.168.1.253:554/cam/realmonitor?channel=1&subtype=1)
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

#[sink1]
#enable=1
#Type - 1=FakeSink 2=EglSink 3=File
#type=2
#sync=0
#source-id=1
#gpu-id=0
#nvbuf-memory-type=0

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=2
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt

[tests]
file-loop=0

• Hardware Platform (Jetson / GPU): Jetson Nano
• DeepStream Version: 5.1
• JetPack Version (valid for Jetson only): 4.5
• TensorRT Version: 7.1.3

May I know how to resolve this issue?

Thanks,
Darshan

#3

Hi,
Since you have 4 live sources, please set

[streammux]
live-source=1
batch-size=4
[primary-gie]
batch-size=4

And enable only [sink0]. Please also execute sudo nvpmodel -m 0 and sudo tegrastats to get max performance. To have more GPU resource in doing deep learning inference, you can set type=5 in [sink0] to use nvoverlaysink plugin.

#4

Hi @DaneLLL,

Thanks for your reply.
I have changed the config file with the modifications suggested by you, but I am getting the same Distortion as I was getting earlier. I am attaching the config file below:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1

[tiled-display]
enable=1
rows=2
columns=2
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=1
num-sources=1
gpu-id=0
camera-width=640
camera-height=480
camera-fps-n=20
camera-fps-d=1
camera-v4l2-dev-node=0
cudadec-memtype=0

[source1]
enable=1
type=4
uri=rtsp://[admin:sl123456@192.168.1.251:554/streaming/channels/102](http://admin:sl123456@192.168.1.251:554/streaming/channels/102)
num-sources=1
gpu-id=0
cudadec-memtype=0

[source2]
enable=1
type=4
uri=rtsp://[admin:admin123@192.168.1.252:554/cam/realmonitor?channel=1&subtype=1](http://admin:admin123@192.168.1.252:554/cam/realmonitor?channel=1&subtype=1)
num-sources=1
gpu-id=0
cudadec-memtype=0

[source3]
enable=1
type=4
uri=rtsp://[admin:admin123@192.168.1.253:554/cam/realmonitor?channel=1&subtype=1](http://admin:admin123@192.168.1.253:554/cam/realmonitor?channel=1&subtype=1)
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

#[sink1]
#enable=1
#Type - 1=FakeSink 2=EglSink 3=File
#type=2
#sync=0
#source-id=1
#gpu-id=0
#nvbuf-memory-type=0

#[sink2]
#enable=1
#Type - 1=FakeSink 2=EglSink 3=File
#type=2
#sync=0
#source-id=2
#gpu-id=0
#nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=1
batch-size=4
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
batch-size=4
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt

[tests]
file-loop=0

Can you please help me to resolve this? It is only happening for the RTSP creamers, not for the Webcam or RPI camera.

IMG_20210323_154548
IMG_20210323_1545484000×3000 2.24 MB

Thanks,
Darshan

#5

Hi @DaneLLL,

Looking forward towards your reply.

Thanks,
Darshan

#6

Hi @DaneLLL ,

Looking forward to your reply.

I also want to know whether it is a Configuration problem, or decoding problem, or Network issue, or a compute capability problem?

Thanks,
Darshan

#7

Hi,
There are 4 sources in your setup. Does the pixel distortion happen to specific source? Or happens to all sources?
And we suggest modify the sample config file source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt to try resnet10 as comparison. To clarify if it happens specific to running Yolov3.

#8

Hi @DaneLLL,

I am getting the distortion again for source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt, with four cameras(3 being RTSP).

Thanks,
Darshan

#9

Hi @DaneLLL,

The Problem is wrt Framerate I had set for the RTSP Cameras. I had set 20 FPS in the Camera Configuration, but on Jetson nano it was only supporting 3 FPS for Individual camera. So, I set 3 FPS for all the 3 RTSP cameras, and there was no Pixel Distortion.

Thanks.

#10

Hi,
From the image, it looks like packets are getting dropped and this is network issue. We would suggest try with single RTSP camera, or try to reduce bitrate, or resolution of the RTSP cameras.

If your source is 20 fps, in [streammux], you would need to adjust

##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000

Default value 40ms is for 30fps. For 20fps, may try 60ms.

#11

Make sure you are using a Gigiabit router.
I had same problem.
This may help.