DS4.0/Nano: uridecodebin (rtsp) "Not supported input Rmsurface" when using ! nvegltransform ! nveglglessink sync=false -e

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

This works fine:

gst-launch-1.0 uridecodebin uri=rtsp://root:admin@169.254.168.156:554/axis-media/media.amp ! nvvideoconvert ! autovideosink

This works too

gst-launch-1.0 uridecodebin uri=file:///home/nvidia/somefile.mp4 ! nvvideoconvert ! nvegltransform ! nveglglessink sync=false -e

This hangs:

gst-launch-1.0 uridecodebin uri=rtsp://root:admin@169.254.168.156:554/axis-media/media.amp ! nvvideoconvert ! nvegltransform ! nveglglessink sync=false -e

with the following messages:

Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Not supported input Rmsurface

2

Probably the same root cause:

This works (file)

gst-launch-1.0 uridecodebin uri=file:///home/nvidia/somefile.mp4 ! nvvideoconvert ! m.sink_0 nvstreammux name=m width=640 height=360 batch-size=1 ! plugin1 ! fakesink

The element plugin1 follows the example plugin in its use of NvBufSurfTransform.

But this fails (rtsp)

gst-launch-1.0 uridecodebin uri=rtsp://root:admin@169.254.168.156:554/axis-media/media.amp ! nvvideoconvert ! m.sink_0 nvstreammux name=m width=640 height=360 batch-size=1 ! plugin1 ! fakesink

With NvBufSurfTransform returning NvBufSurfTransformError_Execution_Error

3

Hi,
Please set GST_DEBUG to know which elements are picked:

export GST_DEBUG=*FACTORY*:4

And compare:
Working pipeline

gst-launch-1.0 uridecodebin uri=rtsp://root:admin@169.254.168.156:554/axis-media/media.amp ! nvvideoconvert ! autovideosink

Failing pipeline

gst-launch-1.0 uridecodebin uri=rtsp://root:admin@169.254.168.156:554/axis-media/media.amp ! nvvideoconvert ! nvegltransform ! nveglglessink sync=false -e
4

Working pipeline:

gst-launch-1.0 uridecodebin uri=rtsp://root:admin@169.254.168.156:554/axis-media/media.amp ! nvvideoconvert ! autovideosink

0:00:00.047010168 9546 0x55aaf62e10 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “uridecodebin”
0:00:00.081090549 9546 0x55aaf62e10 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nvvideoconvert”
0:00:00.086224940 9546 0x55aaf62e10 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “autovideosink”
0:00:00.087771366 9546 0x55aaf62e10 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “fakesink” named “fake-video-sink”
0:00:00.088201645 9546 0x55aaf62e10 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “pipeline”
Setting pipeline to PAUSED …
0:00:00.095159938 9546 0x55aaf62e10 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “nvoverlaysink” named “autovideosink0-actual-sink-nvoverlay”
0:00:00.110772105 9546 0x55aaf62e10 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “rtspsrc” named “source”
0:00:00.112587944 9546 0x55aaf62e10 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtspreal”
0:00:00.113524984 9546 0x55aaf62e10 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtspwms”
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://root:admin@169.254.168.156:554/axis-media/media.amp
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
0:00:00.380877991 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsrc”
0:00:00.383436017 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsrc”
Progress: (request) SETUP stream 0
0:00:00.395390169 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “rtpbin” named “manager”
0:00:00.397255864 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpsession”
0:00:00.398545390 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpssrcdemux”
0:00:00.401403678 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpstorage”
0:00:00.401925895 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “funnel”
0:00:00.402141322 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “funnel”
0:00:00.404356943 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsink”
0:00:00.405899785 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “fakesrc”
0:00:00.406574100 9546 0x55ab02fcf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsink”
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:00.530655567 9546 0x7f6803d540 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpjitterbuffer”
0:00:00.531853315 9546 0x7f6803d540 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpptdemux”
0:00:00.536190795 9546 0x7f4c0038f0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “decodebin”
0:00:00.536579026 9546 0x7f4c0038f0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “typefind” named “typefind”
0:00:00.544249458 9546 0x7f4c0038f0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtph264depay”
0:00:00.546644634 9546 0x7f4c0038f0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “h264parse”
0:00:00.547496328 9546 0x7f4c0038f0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “capsfilter”
0:00:00.552116982 9546 0x7f4c0038f0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nvv4l2decoder”
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261

5

Failing pipeline:

gst-launch-1.0 uridecodebin uri=rtsp://root:admin@169.254.168.156:554/axis-media/media.amp ! nvvideoconvert ! nvegltransform ! nveglglessink sync=false -e

0:00:00.049298150 10175 0x55b2ea4810 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “uridecodebin”
0:00:00.085598881 10175 0x55b2ea4810 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nvvideoconvert”
0:00:00.087127797 10175 0x55b2ea4810 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nvegltransform”
0:00:00.088401638 10175 0x55b2ea4810 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nveglglessink”
0:00:00.088914189 10175 0x55b2ea4810 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “pipeline”
Setting pipeline to PAUSED …

Using winsys: x11
0:00:00.095005725 10175 0x55b2ea4810 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “rtspsrc” named “source”
0:00:00.096758515 10175 0x55b2ea4810 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtspreal”
0:00:00.097593182 10175 0x55b2ea4810 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtspwms”
Pipeline is live and does not need PREROLL …
Got context from element ‘eglglessink0’: gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://root:admin@169.254.168.156:554/axis-media/media.amp
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
0:00:00.363304727 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsrc”
0:00:00.366181789 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsrc”
Progress: (request) SETUP stream 0
0:00:00.376154655 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “rtpbin” named “manager”
0:00:00.378862051 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpsession”
0:00:00.380128307 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpssrcdemux”
0:00:00.383232604 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpstorage”
0:00:00.385259607 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “funnel”
0:00:00.385508122 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “funnel”
0:00:00.388248478 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsink”
0:00:00.389743505 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “fakesrc”
0:00:00.390348825 10175 0x55b2f6ce80 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “udpsink”
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:00.540814348 10175 0x7f7803d8a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpjitterbuffer”
0:00:00.542094525 10175 0x7f7803d8a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpptdemux”
0:00:00.546030096 10175 0x7f64003cf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “decodebin”
0:00:00.546579640 10175 0x7f64003cf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “typefind” named “typefind”
0:00:00.556989184 10175 0x7f64003cf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtph264depay”
0:00:00.559422656 10175 0x7f64003cf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “h264parse”
0:00:00.560230699 10175 0x7f64003cf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “capsfilter”
0:00:00.564805670 10175 0x7f64003cf0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nvv4l2decoder”
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Not supported input Rmsurface

6

Same problem Nano and Xavier (DS 4.0)

7

Hi,
Please check if it helps by specifying format of source pad in nvvideoconvert:

$ gst-launch-1.0 uridecodebin uri=rtsp://127.0.0.1:8554/test ! nvvideoconvert ! <b>'video/x-raw(memory:NVMM),format=NV12'</b> ! nvegltransform ! nveglglessink

We don’t see the issue if launching a rtsp server via test-mp4, but it might help by specifying the format. Please give it a try.

8

Still the same error message. We are using Axis cameras.

9

This works

gst-launch-1.0 uridecodebin uri=rtsp://root:admin@169.254.168.156:554/axis-media/media.amp ! nvvidconv ! ‘video/x-raw,format=NV12’ ! nvvidconv ! nvegltransform ! nveglglessink

The pipeline basically recreates the nvmm frame using nvvidconv at the price of two frame copies.

Note that replacing nvvidconv with nvvideoconvert will not work.

I don’t consider this an acceptable workaround, this is just further diagnosis of the problem.

Will problems like these be fixed?

10

Can anyone provide a list of cameras that DS 4.0 uridecodebin rtsp has been tested with?

11

Modified pipeline based on the workaround:

gst-launch-1.0 uridecodebin uri=rtsp://root:admin@169.254.168.156:554/axis-media/media.amp ! m.sink_0 nvstreammux name=m width=640 height=360 batch-size=1 ! nvvidconv ! video/x-raw,format=NV12 ! nvvideoconvert ! video/x-raw(memory:NVMM),format=NV12 ! plugin1 ! fakesink

Hoping for a fix to avoid frame copies.

12

Hi,
For helping us debugging, please run the test app and share the print:
[url]How to run RTP Camera in deepstream on Nano - DeepStream SDK - NVIDIA Developer Forums

13

Please apply the prebuilt libs and try again:
[url]https://devtalk.nvidia.com/default/topic/1058086/deepstream-sdk/how-to-run-rtp-camera-in-deepstream-on-nano/post/5369676/#5369676[/url]