RGB CSI sensor data encoding

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hello,

I’m developing FPGA parallel to CSI bridge based on Lattice csi2tx reference design. I tried their YUV example project but TX1 didn’t recognize it (packet header multi-bit error). When I tried RGB example everything is working (including v4l-ctl). I want to encode x264, but for this I need to convert BGRA to format supported by encoder (I420,NV12) (I have no idea where RGB888 becomes AR24 (BGRA), because in driver I set Pixel format to RGB888, but v4l-ctl reports:

v4l2-ctl --all
Driver Info (not using libv4l2):
        Driver name   : tegra-video
        Card type     : vi-output, kmtifpga 6-0036
        Bus info      : platform:54080000.vi:0
        Driver version: 4.4.38
        Capabilities  : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
        Width/Height      : 800/480
        Pixel Format      : 'AR24'
        Field             : None
        Bytes per Line    : 3328
        Size Image        : 1597440
        Colorspace        : sRGB
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        Flags             :

Camera Controls

                   frame_length (int)    : min=0 max=32767 step=1 default=1984 value=1984 flags=slider
                    coarse_time (int)    : min=2 max=32761 step=1 default=1978 value=1978 flags=slider
              coarse_time_short (int)    : min=2 max=32761 step=1 default=1978 value=1978 flags=slider
                     group_hold (intmenu): min=0 max=1 default=0 value=0
                     hdr_enable (intmenu): min=0 max=1 default=0 value=0
                       otp_data (str)    : min=0 max=1024 step=2 value='8009057dc0ffffff00c028fbc0ffffff001e00fbc0ffffffc08040020000000090f928fbc0ffffff40b90c00c0ffffffc0f928fbc0fffffff8efb200c0ffffff5898037dc0ffffff8009057dc0ffffff07209a00000000000200000000000000d0f928fbc0ffffff40b90c00c0ffffff00fa28fbc0fffffff8efb200c0ffffff5898037dc0ffffff01000000000000000700000000000000010000000000000020fa28fbc0ffffff9c0d7a00c0ffffff01000000000000005898037dc0fffffff0fa28fbc0ffffff900f7a00c0ffffffc8ff3a01c0ffffff0000000000000000000000000000000000000000000000005898037dc0ffffff000000000000000005209a000000000000000000000000001898037dc0ffffff1812037dc0ffffffb8260801c0ffffff20130801c0ffffff20fb28fb07209a000000000000000000b0fa28fbc0ffffff40b90c00c0ffffffe0fa28fbc0ffffff54901b00c0ffffffd0fa28fbc0ffffff40b90c00c0ffffff00fb28fbc0fffffff8efb200c0ffffff5898037dc0ffffffc89e037dc0ffffff002e057dc0ffffffd0f05f00c0ffffff10fb28fbc0ffffffb4fa7900c0ffffff20fb28fbc0ffffffb0fb7900c0ffffff60fb28fbc0ffffffe4177900c0ffffff0000000000000000c89e037dc0ffffff002e057dc0ffffff208ccdf9c0ffffff48003b01c0ffffff5898037dc0ffffff' flags=read-only, has-payload
                        fuse_id (str)    : min=0 max=16 step=2 value='5898037dc0ffffff' flags=read-only, has-payload
                           gain (int)    : min=256 max=4096 step=1 default=256 value=256 flags=slider
                    bypass_mode (intmenu): min=0 max=1 default=0 value=0
                override_enable (intmenu): min=0 max=1 default=0 value=0
                   height_align (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align (intmenu): min=0 max=2 default=0 value=0
               write_isp_format (bool)   : default=0 value=0

I have tried this pipeline and it works (I get flashing video but I think this is due high frame rate generated by FPGA 130fps and CPU cant convert video fast enough)

gst-launch-1.0 -v v4l2src device=/dev/video0 ! 'video/x-raw, format=(string)BGRA, width=(int)800, height=(int)480, framerate=(fraction)30/1' ! videoconvert ! 'video/x-raw, format=(string)NV12' ! omxh264enc ! qtmux ! filesink location=test.mp4 -ev

So my question is what pipeline should I use for nvvideconv?

2

Hi,
Capability of nvvidconv:

nvidia@tegra-ubuntu:~$ gst-inspect-1.0 nvvidconv

(skip...)

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                 format: { I420, I420_10LE, UYVY, YUY2, YVYU, NV12, GRAY8, BGRx, RGBA }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { I420, UYVY, YUY2, YVYU, NV12, GRAY8, BGRx, RGBA }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                 format: { I420, I420_10LE, UYVY, YUY2, YVYU, NV12, GRAY8, BGRx, RGBA }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { I420, I420_10LE, UYVY, YUY2, YVYU, NV12, GRAY8, BGRx, RGBA }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

(...skip)

You may try

gst-launch-1.0 -v v4l2src device=/dev/video0 ! 'video/x-raw, format=(string)BGRx, width=(int)800, height=(int)480, framerate=(fraction)30/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)NV12' ! omxh264enc ! qtmux ! filesink location=test.mp4 -ev