Nvvideoconvert does not change memory-type if format/resolution is same between input/output?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) dGPU
• DeepStream Version 6.1.1
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only) 515.65.01
• Issue Type( questions, new requirements, bugs) question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

  • pipeline-A : [NV12 pipeline, nvbuf-memory-type=0] → nvvideovonert(nvbuf-memory-type=3) → “capsfilter format=NV12”
  • pipeline-B : [NV12 pipeline, nvbuf-memory-type=0] → nvvideovonert(nvbuf-memory-type=3) → “capsfilter format=RGBA”

pipeline-B output unified memory buffer while pipeline-A does not.
Is it expected behavior?
How to convert memory type without changing format or resolution?

3

How do you confirm whether the memory buffer is NVMM after the nvvideoconvert?

4

@yuweiw
I got nvbufsurface: mapping of memory type (0) not supported error when piepeline-A, while piepeline-B works in my custom app.

How do you confirm whether the memory buffer is NVMM after the nvvideoconvert?

I use caps after videoconvert

  GstElement* capsfilter = gst_element_factory_make("capsfilter", "capsfilter");
  GstCaps* caps = gst_caps_new_simple("video/x-raw", "format", G_TYPE_STRING, "NV12", nullptr);
  GstCapsFeatures* features = gst_caps_features_new("memory:NVMM", nullptr);
  gst_caps_set_features(caps, 0, features);
  g_object_set(capsfilter, "caps", caps, nullptr);
  gst_caps_unref(caps);

Note that the last elemenet before nvvideovonert is streammux(new).

5

I tried the pipeline below, it works fine.

GST_DEBUG=3 gst-launch-1.0 -v videotestsrc ! 'video/x-raw,format=RGBA' !  nvvideoconvert nvbuf-memory-type=3 name=converter1 ! 'video/x-raw(memory:NVMM),format=(string)RGBA' !  smuxer.sink_0 nvstreammux name=smuxer  ! fakesink

Could you attach your 2 whole gst-launch-1.0 commands? Thanks

6

@yuweiw

my app is C/C++.
gst-launch command which imitate the app would be.

export USE_NEW_STREAMMUX=yes

#pipeline-A
gst-launch-1.0 -v videotestsrc ! 'video/x-raw,format=NV12' !  nvvideoconvert nvbuf-memory-type=0 ! 'video/x-raw(memory:NVMM),format=(string)NV12' ! smuxer.sink_0 nvstreammux name=smuxer width=1280 height=720 batch-size=1 ! nvvideoconvert nvbuf-memory-type=3 ! 'video/x-raw(memory:NVMM),format=(string)NV12' ! fakesink

#pipeline-B
gst-launch-1.0 -v videotestsrc ! 'video/x-raw,format=NV12' !  nvvideoconvert nvbuf-memory-type=0 ! 'video/x-raw(memory:NVMM),format=(string)NV12' ! smuxer.sink_0 nvstreammux name=smuxer width=1280 height=720 batch-size=1 ! nvvideoconvert nvbuf-memory-type=3 ! 'video/x-raw(memory:NVMM),format=(string)RGBA' ! fakesink

I belive

  • nvbuf-memory-type at fakesink of pipeline-A is 0
  • nvbuf-memory-type at fakesink of pipeline-B is 3
7

Could you attach your minimized app code? We can verify it from the code. Thanks

8

Understood. I will do it when I have time.

9

Hi @ta.sugiyama , I have confirmed that nvvideoconvert cannot change the memType if the format&resolution is same between input/output. We’ll check this problem.
Also, could you tell us your specific use scenario？

10

@yuweiw

could you tell us your specific use scenario？

12

OK. We do not support this situation with nvvideoconvert at present. We’ll check this probelm and add this to our roadmap. Thanks

13

I got it.
BTW, nvstreammux-new should support nvbuf-memory-type (old one support it)