Nvvideoconvert Memory Compatibility Error

When using deepstream on Jetson devices, a “Memory Compatibility Error” occurs in the nvvideoconvert component, segmentation fault, SIGSEGV or data stream error

• Hardware Platform (Jetson / GPU): Jetson Nano and Jetson NX
• DeepStream Version: 6.0.1 and 6.1
• JetPack Version (valid for Jetson only): 4.6 and 5.0.2
• Issue Type( questions, new requirements, bugs): bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing):

Jetson Nano

  1. docker run -it --rm --gpus=all -v pwd/data:/data nvcr.io/nvidia/deepstream-l4t:6.0.1-base
  2. gst-launch-1.0 -v uridecodebin name=source uri=file:///data/bug.jpg ! nvvideoconvert nvbuf-memory-type=0 ! “video/x-raw(memory:NVMM), format=RGBA” ! muxer.sink_0 nvstreammux name=muxer width=1024 height=1024 batch-size=1 live-source=False ! fakesink

Jetson NX

  1. docker run -it --rm --gpus=all -v pwd/data:/data nvcr.io/nvidia/deepstream-l4t:6.1-base
  2. gst-launch-1.0 -v uridecodebin name=source uri=file:///data/bug.jpg ! nvvideoconvert nvbuf-memory-type=0 ! “video/x-raw(memory:NVMM), format=RGBA” ! muxer.sink_0 nvstreammux name=muxer width=1024 height=1024 batch-size=1 live-source=False ! fakesink

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Log from Nano

Setting pipeline to PAUSED …
Pipeline is PREROLLING …
/GstPipeline:pipeline0/GstURIDecodeBin:source: source = “(GstFileSrc)\ source”
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = image/jpeg, width=(int)780, height=(int)503, sof-marker=(int)0
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstNvJpegDec:nvjpegdec0.GstPad:sink: caps = image/jpeg, width=(int)780, height=(int)503, sof-marker=(int)0
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstNvJpegDec:nvjpegdec0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)RGBA, block-linear=(boolean)false
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)RGBA, block-linear=(boolean)false
/GstPipeline:pipeline0/GstNvStreamMux:muxer.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1024, height=(int)1024, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, framerate=(fraction)0/1, format=(string)RGBA, block-linear=(boolean)false, batch-size=(int)1, num-surfaces-per-frame=(int)1
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1024, height=(int)1024, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, framerate=(fraction)0/1, format=(string)RGBA, block-linear=(boolean)false, batch-size=(int)1, num-surfaces-per-frame=(int)1
/GstPipeline:pipeline0/GstNvStreamMux:muxer.GstNvStreamPad:sink_0: caps = video/x-raw(memory:NVMM), width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)RGBA, block-linear=(boolean)false
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)RGBA, block-linear=(boolean)false
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstURIDecodeBin:source.GstGhostPad:src_0.GstProxyPad:proxypad2: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPad:proxypad1: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
0:00:00.245947034 305 0x55aa6d28a0 WARN nvvideoconvert gstnvvideoconvert.c:3102:gst_nvvideoconvert_transform: error: Memory Compatibility Error:Input surface gpu-id doesnt match with configured gpu-id for element, please allocate input using unified memory, or use same gpu-ids OR, if same gpu-ids are used ensure appropriate Cuda memories are used
0:00:00.246012660 305 0x55aa6d28a0 WARN nvvideoconvert gstnvvideoconvert.c:3102:gst_nvvideoconvert_transform: error: surface-gpu-id=2080560336,nvvideoconvert0-gpu-id=0
0:00:00.246116881 305 0x55aa6d28a0 ERROR nvvideoconvert gstnvvideoconvert.c:3488:gst_nvvideoconvert_transform: buffer transform failed
ERROR: from element /GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0: Memory Compatibility Error:Input surface gpu-id doesnt match with configured gpu-id for element, please allocate input using unified memory, or use same gpu-ids OR, if same gpu-ids are used ensure appropriate Cuda memories are used
Additional debug info:
/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvvideoconvert/gstnvvideoconvert.c(3102): gst_nvvideoconvert_transform (): /GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0:
surface-gpu-id=2080560336,nvvideoconvert0-gpu-id=0
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …
Freeing pipeline …

Log from Jetson NX

Setting pipeline to PAUSED …
Pipeline is PREROLLING …
/GstPipeline:pipeline0/GstURIDecodeBin:source: source = “(GstFileSrc)\ source”
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = image/jpeg, width=(int)780, height=(int)503, sof-marker=(int)0
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstNvJpegDec:nvjpegdec0.GstPad:sink: caps = image/jpeg, width=(int)780, height=(int)503, sof-marker=(int)0
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstNvJpegDec:nvjpegdec0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)Y444, width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstTypeFindElement:typefind: Internal data stream error.
Additional debug info:
gsttypefindelement.c(1228): gst_type_find_element_loop (): /GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstTypeFindElement:typefind:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …
Freeing pipeline …
root@6c1e559e058c:/opt/nvidia/deepstream/deepstream-6.1# gst-launch-1.0 -v uridecodebin name=source uri=file:///data/bug.jpg ! nvvideoconvert nvbuf-memory-type=0 ! “video/x-raw(memory:NVMM), format=RGBA” ! fakesink
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
/GstPipeline:pipeline0/GstURIDecodeBin:source: source = “(GstFileSrc)\ source”
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = image/jpeg, width=(int)780, height=(int)503, sof-marker=(int)0
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstNvJpegDec:nvjpegdec0.GstPad:sink: caps = image/jpeg, width=(int)780, height=(int)503, sof-marker=(int)0
/GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstNvJpegDec:nvjpegdec0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)Y444, width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstTypeFindElement:typefind: Internal data stream error.
Additional debug info:
gsttypefindelement.c(1228): gst_type_find_element_loop (): /GstPipeline:pipeline0/GstURIDecodeBin:source/GstDecodeBin:decodebin0/GstTypeFindElement:typefind:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …
Freeing pipeline …

I tried changing the resolution of the image, but it only changed the error.

test_images.zip (121.8 KB)

I am checking.

this issue can reproduce on xavier deepstream6.1, will continue to check.
command: gst-launch-1.0 -v uridecodebin name=source uri=file:///home/nvidia/rec/test_images/bug_768x480.jpg ! nvvideoconvert ! “video/x-raw(memory:NVMM), format=RGBA” ! fakesink
result:
crash.txt (5.0 KB)

In all, NV hardware decoding dose not support these jpg files, you can use software decoding, like this: gst-launch-1.0 filesrc location=bug_768x480.jpg ! jpegdec ! nvvideoconvert ! “video/x-raw(memory:NVMM), format=RGBA” ! fakesink,
here is the detail:

  1. about bug.jpg, nvjpegdec dose not support this jpg, here is the key log:
    0:00:09.021395353 e[334m378315e[00m 0xaaaaf39a3580 e[37mDEBUG e[00m e[00m nvvideoconvert gstnvvideoconvert.c:2352:gst_nvvideoconvert_accept_caps:e[00m could not transform video/x-raw(memory:NVMM), format=(string)Y444, width=(int)780, height=(int)503, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1 in anything we support
  2. about bug_768x480.jpg and bug_1280x720.log, nvjpegdec decoded failed because of Application transferred too few scanlines, here is the key log:
    0:00:07.937924969 378412 0xaaaaec3e6980 WARN videodecoder gstvideodecoder.c:4155:_gst_video_decoder_error: error: Failed to decode JPEG image
    0:00:07.937969833 378412 0xaaaaec3e6980 WARN videodecoder gstvideodecoder.c:4157:_gst_video_decoder_error: error: Decode error #69: Application transferred too few scanlines

Thank you. We downgraded nvjpegdec rank for Jetson when using decodebin.

Glad to know you fixed it, thanks for the update!