Nvv4l2camerasrc is producing an incorrect image on Orin

[rmandala]

Hi,

I am trying to use nvv4l2camerasrc plugin but it is giving the incorrect image output. Is there any setting I am missing here?

Here is the Gstreamer command.

gst-launch-1.0 -v nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),width=1280,height=720" !  nvvidconv ! video/x-raw,format=RGBA ! fpsdisplaysink video-sink=autovideosink sync=false

On the other hand, I can get a proper output image with v4l2src.

gst-launch-1.0 -v v4l2src device=/dev/video0 ! "video/x-raw,width=1280,height=720" !  videoconvert ! fpsdisplaysink video-sink=autovideosink sync=false

[enrique.ramirez]

Hi @rmandala,

Maybe the RGBA format used is not the right one.
Could you check the caps negotiated by videoconvert and fpsdisplaysink on the working pipe? You can check that by adding -v to the end of the pipe.

Also, you can try this pipe to see if you get the right colors:

gst-launch-1.0 -v nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),width=1280,height=720" ! nvvidconv ! "video/x-raw(memory:NVMM), format=I420" ! nvoverlaysink sync=false

Best Regards,

Enrique Ramirez
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.com
Website: www.ridgerun.com

[rmandala]

Hi Enrique,

There is no nvoverlaysink plugin on Orin Nano running JP 6.1. I tried removing the src caps from nvvidconv and still seeing the same incorrect image as output. I found something useful here, v4l2src is preceiving the video as YUY2, pixel format whereas nvv4l2camerasrc as UYVY pixel format both are YUYV 422 formats.

When tried to set the pixel format as YUY2 for nvv4l32camerasrc then getting below warning which means that it can not handle that frame format.
WARNING: erroneous pipeline: could not link nvv4l2camerasrc0 to nvvconv0, nvv4l2camerasrc0 can’t handle caps video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)YUY2

I suspect that the source plugin acquiring data in the form of UYVY instead of YUYV is causing color swap in the output image.

Incorrect image pipeline:
src plugin:
caps = video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)UYVY, interlace-mode=(string)progressive, framerate=(fraction)30/1

videoconvert:
caps = video/x-raw, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, framerate=(fraction)30/1, format=(string)RGBA

fpsdisplaysink:
caps = video/x-raw(memory:SystemMemory, meta:GstVideoOverlayComposition), width=(int)1280, height=(int)720, interlace-mode=(string)progressive, framerate=(fraction)30/1, format=(string)RGBA

Correct image pipline:
src plugin:
caps = video/x-raw, width=(int)1280, height=(int)720, format=(string)YUY2, framerate=(fraction)10/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:4:5:1

videoconvert:
caps = video/x-raw, width=(int)1280, height=(int)720, format=(string)YUY2, framerate=(fraction)10/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:4:5:1

fpsdisplaysink:
caps = video/x-raw, width=(int)1280, height=(int)720, format=(string)YUY2, framerate=(fraction)10/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:4:5:1

Hardware information:
NVIDIA NVIDIA Jetson Orin Nano Developer Kit
L4T 36.3.0 [ JetPack 6.0]
Ubuntu 22.04.4 LTS
Kernel Version: 5.15.136-tegra
CUDA 12.2.140
CUDA Architecture: 8.7
OpenCV version: 4.8.0
OpenCV Cuda: NO
CUDNN: 8.9.4.25
TensorRT: 8.6.2.3
Vision Works: NOT_INSTALLED
VPI: 3.1.5
Vulcan:

[enrique.ramirez]

Hi @rmandala

Inspecting the nvv4l2camerasrc element, it seems that only the UYVY format is supported:

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
                 format: { (string)UYVY }
         interlace-mode: { (string)progressive, (string)interlaced }
              framerate: [ 0/1, 2147483647/1 ]

I just confirmed by checking the documentation:
https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3275/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/accelerated_gstreamer.html#wwpID0E0BR0HA

[rmandala]

Thanks, Enrique for the info. Perhaps the plugin should throw an error instead of reading the YUYV data as UYVY data. This causes confusion while using the nvv4l2camerasrc.