Nvvideoconvert gives error when input buffer is RGB, BGR or UYVP

hyperlight · October 31, 2022, 11:30pm

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson AGX Xavier Dev
• DeepStream Version: 6.1.1
• JetPack Version (valid for Jetson only): 5.0.2
• TensorRT Version: 8.4
• Issue Type( questions, new requirements, bugs): bug
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

This pipeline gives error:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw, width=640, height=480, framerate=30/1" ! videoconvert ! "video/x-raw,format=(string)RGB" ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=(string)NV12" ! cam1.sink_0 nvstreammux name=cam1 width=1024 height = 1200 batch-size=1 ! nvvideoconvert ! videoconvert ! xvimagesink

The error: nvvideoconvert gstnvvideoconvert.c:3750:gst_nvvideoconvert_transform: buffer transform failed

Full output:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)RGB
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)RGB
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)NV12, block-linear=(boolean)false
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)NV12, block-linear=(boolean)false
/GstPipeline:pipeline0/GstNvStreamMux:m.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1024, height=(int)1200, framerate=(fraction)30/1, format=(string)NV12, block-linear=(boolean)false, batch-size=(int)1, num-surfaces-per-frame=(int)1
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert1.GstPad:src: caps = video/x-raw, width=(int)1024, height=(int)1200, framerate=(fraction)30/1, batch-size=(int)1, num-surfaces-per-frame=(int)1, format=(string)NV12, block-linear=(boolean)false
/GstPipeline:pipeline0/GstVideoConvert:videoconvert1.GstPad:src: caps = video/x-raw, width=(int)1024, height=(int)1200, framerate=(fraction)30/1, batch-size=(int)1, num-surfaces-per-frame=(int)1, block-linear=(boolean)false, format=(string)YV12
/GstPipeline:pipeline0/GstXvImageSink:xvimagesink0.GstPad:sink: caps = video/x-raw, width=(int)1024, height=(int)1200, framerate=(fraction)30/1, batch-size=(int)1, num-surfaces-per-frame=(int)1, block-linear=(boolean)false, format=(string)YV12
/GstPipeline:pipeline0/GstVideoConvert:videoconvert1.GstPad:sink: caps = video/x-raw, width=(int)1024, height=(int)1200, framerate=(fraction)30/1, batch-size=(int)1, num-surfaces-per-frame=(int)1, format=(string)NV12, block-linear=(boolean)false
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1024, height=(int)1200, framerate=(fraction)30/1, format=(string)NV12, block-linear=(boolean)false, batch-size=(int)1, num-surfaces-per-frame=(int)1
/GstPipeline:pipeline0/GstNvStreamMux:m.GstNvStreamPad:sink_0: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)NV12, block-linear=(boolean)false
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)NV12, block-linear=(boolean)false
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)RGB
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)RGB
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)YUY2, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:5:1, interlace-mode=(string)progressive
0:00:02.740203784  6008 0xaaaabfaf85e0 ERROR         nvvideoconvert gstnvvideoconvert.c:3750:gst_nvvideoconvert_transform: buffer transform failed
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason error (-5)
Execution ended after 0:00:01.864619813
Setting pipeline to NULL ...
Freeing pipeline ...

To fix the error, we need to change to colour format of the input buffer of nvvideoconvert. Here I use NV12, but it can be any format supported by nvvideoconvert and videoconvert other than RGB, BGR and UYVP:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw, width=640, height=480, framerate=30/1" ! videoconvert ! "video/x-raw,format=(string)NV12" ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=(string)NV12" ! cam1.sink_0 nvstreammux name=cam1 width=1024 height = 1200 batch-size=1 ! nvvideoconvert ! videoconvert ! xvimagesink

yingliu · November 1, 2022, 1:38am

From the output of “gst-inspect-1.0 nvvideoconvert” you can see below format supported for video-raw:
(string)I420, (string)P010_10LE, (string)NV12, (string)BGRx, (string)RGBA, (string)GRAY8, (string)YUY2, (string)UYVY, (string)YVYU, (string)Y42B, (string)RGB, (string)BGR, (string)UYVP
As there is videoconvert in your pipeline, you can let videoconvert to connect with nvvideoconvert directly and let the elements negotiate the capacities, the capacities filter in between is not necessary.

hyperlight · November 1, 2022, 2:22am

@yingliu,

Thank you for the reply. I use v4l2src to reproduce the bug with any camera that support v4l2src, the capfilter added between videoconvert and nvvideoconvert is to demonstrate that nvvideoconvert gives error for some colour formats that it supports. My actual use case needs RGB as the colour format to nvvideoconvert. At the moment, I must use videoconvert just to convert from RGB to another colour format accepted by nvvideoconvert even though nvvideoconvert supports RGB directly. The post is to report a potential bug. Can you confirm if it’s indeed a bug or something else?

fanzh · November 1, 2022, 3:31pm

can you provide more logs? thanks, please do “export GST_DEBUG=5” first, then run again.

hyperlight · November 1, 2022, 5:28pm

@fanzh,

Thank you for your reply. Here is the log files for these settings:

export GST_DEBUG=5
export GST_DEBUG_NO_COLOR=1
export GST_DEBUG_FILE=/tmp/log1.txt

Command that cause error (pipeline stopped due to error):

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw, width=640, height=480, framerate=30/1" ! videoconvert ! "video/x-raw,format=(string)RGB" ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=(string)NV12" ! cam1.sink_0 nvstreammux name=cam1 width=1024 height = 1200 batch-size=1 ! nvvideoconvert ! videoconvert ! xvimagesink

log1.txt (6.0 MB)

Command that ran normally (I closed the display window to stop the pipeline):

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw, width=640, height=480, framerate=30/1" ! videoconvert ! "video/x-raw,format=(string)NV12" ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=(string)NV12" ! cam1.sink_0 nvstreammux name=cam1 width=1024 height = 1200 batch-size=1 ! nvvideoconvert ! videoconvert ! xvimagesink

log2.txt (32.3 MB)

yuweiw · November 3, 2022, 5:24am

Hi @hyperlight , please set the compute-hw=1 para to nvvideoconvert with DS6.1.1 when you change RGB format to YUV format. Thanks

! nvvideoconvert compute-hw=1 !

system · November 21, 2022, 7:28pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.