DeepStream on Jetson AGX Xavier - Jetpack 4.3 installed with sdkmanager

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi,
I am trying to:

  1. rtsp out a CSI camera from Xavier to estimate if it will work over the network with the highest possible performance.
/opt/nvidia/deepstream/deepstream-4.0/samples/configs/deepstream-app$ deepstream-app -c source1_csi_dec_infer_resnet_int8.txt

where is the file I enabled the sink 2 and it generated url:rtsp://localhost:8554/ds-test

but both

vlc rtsp://localhost:8554/ds-test

and

gst-launch-1.0 rtspsrc location=rtsp://localhost:8554/ds-test

will throw exceptions, e.g:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://localhost:8554/ds-test
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc1: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc1:
streaming stopped, reason not-linked (-1)
Execution ended after 0:00:02.016119097
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

what plays somehow but with terrific image that fades away every few seconds is:

mplayer rtsp://localhost:8554/ds-test

7f7a2bbab0]error while decoding MB 34 44
[h264 @ 0x7f7a2bbab0]concealing 95 DC, 95 AC, 95 MV errors in P frame
V:  12.8   0/  0  7%  0%  0.0% 0 0 
[rtsp @ 0x7f7ad28e60]max delay reached. need to consume packet
[rtsp @ 0x7f7ad28e60]RTP: missed 668 packets
[h264 @ 0x7f7a2bbab0]corrupted macroblock 52 44 (total_coeff=-1)
[h264 @ 0x7f7a2bbab0]error while decoding MB 52 44
[h264 @ 0x7f7a2bbab0]concealing 77 DC, 77 AC, 77 MV errors in P frame
V:  14.9   0/  0  7%  0%  0.0% 0 0 
[rtsp @ 0x7f7ad28e60]max delay reached. need to consume packet
[rtsp @ 0x7f7ad28e60]RTP: missed 659 packets
[h264 @ 0x7f7a2bbab0]concealing 2055 DC, 2055 AC, 2055 MV errors in I frame
V:  17.0   0/  0  6%  0%  0.0% 0 0 
[rtsp @ 0x7f7ad28e60]max delay reached. need to consume packet
[rtsp @ 0x7f7ad28e60]RTP: missed 657 packets
[h264 @ 0x7f7a2bbab0]corrupted macroblock 25 43 (total_coeff=-1)
[h264 @ 0x7f7a2bbab0]error while decoding MB 25 43

am I missing some settings adjustment?

2

Hi,

This could be an issue of codec or bitrate,
Can you share the relevant sink section of the config file?

Here is an example:
[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=4
#1=h264 2=h265
codec=1
sync=0
bitrate=4000000

set below properties in case of RTSPStreaming

rtsp-port=8554
udp-port=5400

Are you sure the RTSP port is not used by another source?

Thanks

3

Can you share the relevant sink section of the config file?

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP 5=CSI
type=5
camera-width=1280
camera-height=720
camera-fps-n=30
camera-fps-d=1
[sink0]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=5
sync=0
display-id=0
offset-x=0
offset-y=0
width=0
height=0
overlay-id=1
source-id=0

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4
codec=1
sync=0
bitrate=2000000
output-file=out.mp4
source-id=0

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=4
#1=h264 2=h265
codec=1
sync=0
bitrate=4000000
# set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400

Are you sure the RTSP port is not used by another source?
Yes, it seems so that it is not used by anything

4

Hi,
Please use another PC as client. For example, a laptop with Windows OS or desktop PC with Linux OS. You can open network stream through vlc player. The uri is

rtsp://_XAVIER_IP_ADDRESS_:8554/ds-test
5

it turned out that from the same computer it works with the execution of the code below:

$ gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/ds-test ! queue ! decodebin ! nvvidconv ! videoconvert ! xvimagesink

However, network execution from host pc with VLC crashed and neither windows couldn’t play and connect to the stream with vlc. Maybe it is an issue with vlc though.
I am just wondering if there is nano Rpi deepstream conector/agent or what is the way to deliver rtsp stream from nano device in order to be processed at Xavier? gstreamer generated pipeline perhaps?

6

Hi,
To run server/client on Xavier, we can run the following client pipeline:

$ gst-launch-1.0 uridecodebin uri=rtsp://127.0.0.1:8554/ds-test ! nvoverlaysink sync=false

Please give it a try.

Not quire understand what this usecase is. Please describe the usecase in detail. Use Nano as server and Raspberry Pi as client, probably?

7

the idea was to use Xavier as a deepstream server,
and use also jetson nano with RPi camera [CSI sensor that is the default for the devkit] to generate a stream that will be processed at Xavier in Deepstream;
I know that nano supports deepstream to a certain extent and I was just wondering if it would make sense to utilize deepstream components at the nano in order to get the stream from it generated from camera and delivered to the Xavier somehow; Otherwise gstreamer pipeline with ./test-launch could be used for the purpose as before introducing deepstream for nano.
The detailed idea was to start streaming with rtsp gstreamer at nano device and estimate if it will perform realtime via the network if playing the stream at Xavier device; Playing the stream at Xavier device with Deepstream; In order to see if at all 4k rtsp stream can be processed over the network realtime, in order to allow teleoperation of a facility vehicle [ toy car] by a remote human operator.

8

Hi,
This should work. You can run test-launch on Jetson Nano:

./test-launch "nvarguscamerasrc ! video/x-raw(memory:NVMM), format=NV12, width=1920, height=1080, framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM), width=640, height=480, format=NV12, framerate=30/1 ! omxh265enc ! rtph265pay name=pay0 pt=96 config-interval=1"

And run deepstream-app on Xavier. There is a reference for modifying config file to RTSP source:
https://devtalk.nvidia.com/default/topic/1058086/deepstream-sdk/how-to-run-rtp-camera-in-deepstream-on-nano/post/5366807/#5366807
127.0.0.1 in above link should be modified to IP address of Jetson Nano.

9

Thank you for sharing;
Maybe you have any idea why in the latest 4.3 Jetpack based release DeepStream won’t show vide output if using usb-c monitor?
neither regular generic gstreamer that uses smth like

gst-launch-1.0 videotestsrc ! nvvidconv ! nvoverlaysink

which neither show video stream.
In previous Jetpack releases, the issue did not exist.

10

Hi Andrey1984,

Please try below pipeline:

$ gst-launch-1.0 videotestsrc ! nvvidconv ! nvoverlaysink <b>display-id=2</b>
11

thank you; it worked

12

how do I pass the

display-id=2

parameter when I am running deepstream at Xavier?

deepstream-app -c source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt

?

13

Hi,
You need to modify the source code and rebuild deepstream-app. The code is in

deepstream_sdk_v4.0.2_jetson\sources\apps\apps-common\src\deepstream_sink_bin.c

Adding gst_object_set(nvoverlaysink, “display-id”, 2, NULL); should work.
It is implemented and can be set in config file.

14

I located the file:

/opt/nvidia/deepstream/deepstream-4.0/sources/apps/apps-common/src/deepstream_sink_bin.c

could you extend?[
how to rebuild the deepstream app from performed by Jetpack 4.3 installation that included Deepstream?
Or I just rather download sources, edit the file and install it, as usual, running the ./install.sh script:

Untar binaries.tbz2:
   sudo tar -xvf binaries.tbz2 -C /

6. Run the install.sh script as follows:
   sudo ./install.sh

7. Execute the following command on the Jetson development board:
   sudo ldconfig

adding

gst_object_set(nvoverlaysink, "display-id", 2, NULL);

to any line? at the beginning? at the end of the file?
like that?
I downloaded and unpacked the source code:

#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>

#include "deepstream_common.h"
#include "deepstream_sinks.h"
#include <gst/rtsp-server/rtsp-server.h>

static guint uid = 0;

GST_DEBUG_CATEGORY_EXTERN (NVDS_APP);
gst_object_set(nvoverlaysink, "display-id", 2, NULL);
/**
 * Function to create sink bin for Display / Fakesink.
 */
static gboolean

probably capital letters:

[GST_OBJECT_SET(nvoverlaysink, "display-id", 2, NULL);

or rather somewhere to this section?

case NV_DS_SINK_RENDER_OVERLAY:
#ifndef IS_TEGRA
      NVGSTDS_ERR_MSG_V ("Overlay is only supported for Jetson");
      return FALSE;
#endif
      GST_CAT_INFO (NVDS_APP, "NVvideo renderer\n");
      bin->sink = gst_element_factory_make (NVDS_ELEM_SINK_OVERLAY, elem_name);
      <b>g_object_set (G_OBJECT (bin->sink), "display-id", config->display_id,</b>
          NULL);
      g_object_set (G_OBJECT (bin->sink), "overlay", config->overlay_id, NULL);
      g_object_set (G_OBJECT (bin->sink), "overlay-x", config->offset_x, NULL);
      g_object_set (G_OBJECT (bin->sink), "overlay-y", config->offset_y, NULL);
      g_object_set (G_OBJECT (bin->sink), "overlay-w", config->width, NULL);
      g_object_set (G_OBJECT (bin->sink), "overlay-h", config->height, NULL);
      break;
    case NV_DS_SINK_FAKE:

I tried to get it built with install.sh after removing /opt/nvidia/deepstream, but probably I am missing something as it remained the same situation without video output
Does the line below imply that display_id can be passed from config file?

g_object_set (G_OBJECT (bin->sink), "display-id", config->display_id
15

Hi,
https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html#page/DeepStream%2520Development%2520Guide%2Fdeepstream_app_config.3.2.html%23wwpID0ENHA
Yes, you are right. It is implemented and you can set it in config file. My bad that I missed it. Thanks for pointing it out.

16

Hi DaneLLL,
Thank you for pointing out the corresponding section in the Documentation! I wouldn’t find it, but for your help.
Bravo!
code below turned out to work:

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=5
sync=0
display-id=2
offset-x=0
offset-y=0
width=0
height=0
overlay-id=1
source-id=0

however after execution I can not stop it neither with contrl-c nor with other keypresses; It will show full scren video output and won’t close or scale.

17

Hi Andrey1984,

We can’t reproduce your issue.
Set the same config with yours and using ‘q’ or ctrl+c to quitting are success.
The display-id between 0-2, we tried display-id=1 is working.

18

it got to work eventually with “q” and cntrl+c
Thanks