Nvvideoconvert report error when working with a customized plugin

Hello, Nvidia’s developer:

We made a customized “nvarguscameras” plugin, based on the code of syncSensor and nvarguscamerasrc, which open the two CSI cameras, capture the frames pair, and output the batched buffer, trying to be same as the output of “nvstreammux”.

Below is the test results and issues:

  1. Below pipeline works, which means the “nvmultistreamtilter” can access the batched buffer and process the frames correctly:

gst-launch-1.0 -e nvarguscameras ! “video/x-raw(memory:NVMM),format=(string)NV12,width=(int)1920,height=(int)1080,framerate=(fraction)30/1” ! nvmultistreamtiler width=3840 height=1080 rows=1 columns=2 ! nvvideoconvert ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvv4l2h264enc bitrate=8000000 ! h264parse ! qtmux ! filesink location=“test.mp4” sync=0 async=0

  1. Below pipeline workds too, which means the “nvstreamdemux” can access the batched buffer and demux it correctly:

gst-launch-1.0 -e nvarguscameras silent=false ! “video/x-raw(memory:NVMM),format=(string)RGBA,width=(int)1920,height=(int)1080,framerate=(fraction)30/1” ! nvstreamdemux name=d d.src_0 ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),format=(string)I420’ ! nvv4l2h264enc bitrate=8000000 ! h264parse ! qtmux ! filesink location=“/home/camera/reeplayer/test1.mp4” sync=0 async=0 d.src_1 ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),format=(string)I420’ ! nvv4l2h264enc bitrate=8000000 ! h264parse ! qtmux ! filesink location=“/home/camera/reeplayer/test2.mp4” sync=0 async=0

  1. Below pipeline works too, which means the “nvdsprepprocess” can access the batched buffer and process the frames correctly:

gst-launch-1.0 -e nvarguscameras ! “video/x-raw(memory:NVMM),format=(string)NV12,width=(int)1920,height=(int)1080,framerate=(fraction)30/1” ! nvdspreprocess config-file= /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvdspreprocess/config_preprocess.txt ! nvmultistreamtiler width=3840 height=1080 rows=1 columns=2 ! nvvideoconvert ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvv4l2h264enc bitrate=8000000 ! h264parse ! qtmux ! filesink location=“test.mp4” sync=0 async=0

  1. But below pipeline raised an error from “nvvideoconvert”, which indicates the “nvvideoconvert” found an issues with the batched buffer:

gst-launch-1.0 nvarguscameras ! “video/x-raw(memory:NVMM),format=(string)NV12,width=(int)1920,height=(int)1080,framerate=(fraction)30/1” ! nvvideoconvert ! ‘video/x-raw(memory:NVMM), format=(string)RGBA’ ! nvmultistreamtiler width=3840 height=1080 rows=1 columns=2 ! nvvideoconvert ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvv4l2h264enc bitrate=8000000 ! h264parse ! qtmux ! filesink location=“test.mp4” sync=0 async=0 -e

The error info:

ERROR nvvideoconvert gstnvvideoconvert.c:3484:gst_nvvideoconvert_transform: buffer transform failed

  1. Below pipeline works again, which means the batched buffer from our plugin could be “demuxed” and then “muxed” again correctly, the re-muxed batched buffer has no problem to be consumed by the “nvvideoconvert”:

gst-launch-1.0 -e nvarguscameras silent=false ! “video/x-raw(memory:NVMM),format=(string)NV12,width=(int)1920,height=(int)1080,framerate=(fraction)30/1” ! nvstreamdemux name=d nvstreammux batch-size=2 width=1920 height=1080 name=m ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),format=(string)RGBA’ ! nvmultistreamtiler width=3840 height=1080 rows=1 columns=2 ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),format=(string)I420’ ! nvv4l2h264enc bitrate=8000000 ! h264parse ! qtmux ! filesink location=“/home/camera/reeplayer/test.mp4” sync=0 async=0 d.src_0 ! m.sink_0 d.src_1 ! m.sink_1

Above test results shown that the “nvvideoconvert” only raised the error when it is connected to the “nvarguscameras” directly. And the batched buffer could be consumed correctly by some other deepstream plugins. So we guess (1). it is possible that our data/meta in the batched buffer has a problem that only critical to the processing in “nvvideoconvert”. or (2). there is a bug in “nvvideoconvert” when it process the batched buffer. We think the possibility of (1) is higher, but we have no source code of “nvvideoconvert” to debug and figure out what in the batched buffer triggered the error.

It is not convenience at this moment to public our source code (the client will open source after the project is done), so I attached the binary file of the “nvarguscameras”. There should no problem for you to load it with the same environment (refer to the details below). We hope you can help us to debug it with the source code of “nvvideoconvert”, to figure out what triggered the error in “nvvideoconvert”.

Please let me know if you have issues to reproduce the error. Or you may provide a private way so that we can share the source code to you. Thanks.


libgstnvarguscameras.so (369.0 KB)

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson Xavier NX developer kit with two CSI camera sensors

• DeepStream Version
DeepStream 6.0.0

• JetPack Version (valid for Jetson only)
JetPack 6.0

• TensorRT Version
Default

• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
bugs

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Hi,
Since the implmentaion of nvvideoconvert plugin is based on NvBufSurface functions in

/opt/nvidia/deepstream/deepstream-6.0/sources/includes/nvbufsurftransform.h

We suggest implement the functions you need nvvideoconvert to do in the nvarguscameras. This should be do-able since NvBufSurface functions are all public.

Thanks for your suggestion. Yes we have done that in our code (e.g. convert the NV12 to RGBA and flip the image), and it works well. But that does not mean the problem is gone. The bug still exists in our code or in the “nvvideoconvert” code. I just hope we can collaborate to remove the bug to avoid potential issues, if you have time to work on it.

Hi,
Please check if the value is not identical in NvBufSurce:

  /** Holds the batch size. */
  uint32_t batchSize;
  /** Holds the number valid and filled buffers. Initialized to zero when
   an instance of the structure is created. */
  uint32_t numFilled;

And can print out width,height,bufferDesc of each buffer in surfaceList to make sure each NvBufSurface is filled in correctly.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.