TX2 VI mode and ISP mode bugs when using CSI2 virtual channels

agustg650e · July 29, 2021, 12:06pm

I’m using Jetson TX2 SoM with L4T 32.4.3, JetPack 4.4 and two OV9782 cameras connected via
2x DS90UB953-Q1 and DS90UB954-Q1 FPD-Link III deserializer to the CSI-A/CSI-B ports, using
4 data lanes and one clock lane. This requires using CSI2 virtual channels, so I’m using a configuration
with virtual channels VC0 and VC1.

The OV9782 camera driver was developed by Nvidia partner D3 Engineering and works without issues,
I can stream from both cameras simultaneously either as raw images with v4l2-ctl tool or in ISP mode
(using Argus/GStreamer). Below some more info about the cameras/interfaces:

$ media-ctl -p
Media controller API version 0.1.0

Media device information
------------------------
driver          tegra-vi4
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  0.0.0

Device topology
- entity 1: 150c0000.nvcsi--6 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "ov9782d3 9-0060":0 [ENABLED]
	pad1: Source
		-> "vi-output, ov9782d3 9-0060":0 [ENABLED]

- entity 4: 150c0000.nvcsi--5 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
	pad0: Sink
		<- "ov9782d3 10-0060":0 [ENABLED]
	pad1: Source
		-> "vi-output, ov9782d3 10-0060":0 [ENABLED]

- entity 7: ov9782d3 10-0060 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev2
	pad0: Source
		[fmt:SBGGR10_1X10/1280x800 field:none colorspace:srgb]
		-> "150c0000.nvcsi--5":0 [ENABLED]

- entity 9: vi-output, ov9782d3 10-0060 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
	pad0: Sink
		<- "150c0000.nvcsi--5":1 [ENABLED]

- entity 23: ov9782d3 9-0060 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
	pad0: Source
		[fmt:SBGGR10_1X10/1280x800 field:none colorspace:srgb]
		-> "150c0000.nvcsi--6":0 [ENABLED]

- entity 25: vi-output, ov9782d3 9-0060 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Sink
		<- "150c0000.nvcsi--6":1 [ENABLED]

$  argus_oneshot -l
Executing Argus Sample: argus_oneshot
2 Available CameraDevices:

  ==== CameraDevice 0: =========================================
    UUID:                      adca2c00,0f01,11e5,0000,00,00,00,00,00,00
    MaxAeRegions:              64
    MaxAwbRegions:             64
    FocusPositionRange:        [0, 0]
    LensApertureRange:         [2.970000, 2.970000]
    IspDigitalGainRange:       [1.000000, 256.000000]
    ExposureCompensationRange: [-2.000000, 2.000000]
    NumSensorModes:            2
    SensorMode 0:
        Resolution:         1280x800
        ExposureTimeRange:  [9000, 10000000]
        FrameDurationRange: [50000000, 1000001024]
                            (1.00 to 20.00 fps)
        AnalogGainRange:    [1.000000, 15.937500]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 1:
        Resolution:         1280x800
        ExposureTimeRange:  [68000, 20000000]
        FrameDurationRange: [20000000, 1000001024]
                            (1.00 to 50.00 fps)
        AnalogGainRange:    [1.000000, 255.000000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No

  ==== CameraDevice 1: =========================================
    UUID:                      adca2c00,0f01,11e5,0001,00,00,00,00,00,00
    MaxAeRegions:              64
    MaxAwbRegions:             64
    FocusPositionRange:        [0, 0]
    LensApertureRange:         [2.970000, 2.970000]
    IspDigitalGainRange:       [1.000000, 256.000000]
    ExposureCompensationRange: [-2.000000, 2.000000]
    NumSensorModes:            2
    SensorMode 0:
        Resolution:         1280x800
        ExposureTimeRange:  [9000, 10000000]
        FrameDurationRange: [50000000, 1000001024]
                            (1.00 to 20.00 fps)
        AnalogGainRange:    [1.000000, 15.937500]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No
    SensorMode 1:
        Resolution:         1280x800
        ExposureTimeRange:  [68000, 20000000]
        FrameDurationRange: [20000000, 1000001024]
                            (1.00 to 50.00 fps)
        AnalogGainRange:    [1.000000, 255.000000]
        InputBitDepth:      10
        OutputBitDepth:     10
        SensorModeType:     SENSOR_MODE_TYPE_BAYER
        IS WDR Mode: No

Two simultaneous streams in VI mode work as expected, i.e.:

$ v4l2-ctl -d /dev/video0 -c bypass_mode=0 --stream-mmap --stream-count=1000
<<<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<<<< 20.14 fps
...

$ v4l2-ctl -d /dev/video1 -c bypass_mode=0 --stream-mmap --stream-count=1000
<<<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<<<< 20.14 fps
...

I can also stop and restart one stream (VC0 or VC1) while another stream (VC1 or VC0) is active.

Two simultaneous streams in ISP mode also work as expected, i.e.:

$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! "video/x-raw(memory:NVMM),width=1280,height=800,format=NV12,framerate=20/1" ! nvv4l2h264enc ! h264parse ! rtph264pay mtu=1400 ! udpsink host=192.168.10.30 port=5000
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1280 x 800 FR = 20,000000 fps Duration = 50000000 ; Analog Gain range min 1,000000, max 15,937500; Exposure Range min 9000, max 10000000;

GST_ARGUS: 1280 x 800 FR = 50,000000 fps Duration = 20000000 ; Analog Gain range min 1,000000, max 255,000000; Exposure Range min 68000, max 20000000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 0
   Output Stream W = 1280 H = 800
   seconds to Run    = 0
   Frame Rate = 20,000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
H264: Profile = 66, Level = 0
...
$ gst-launch-1.0 nvarguscamerasrc sensor-id=1 sensor-mode=0 ! "video/x-raw(memory:NVMM),width=1280,height=800,format=NV12,framerate=20/1" ! nvv4l2h264enc ! h264parse ! rtph264pay mtu=1400 ! udpsink host=192.168.10.30 port=5001
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1280 x 800 FR = 20,000000 fps Duration = 50000000 ; Analog Gain range min 1,000000, max 15,937500; Exposure Range min 9000, max 10000000;

GST_ARGUS: 1280 x 800 FR = 50,000000 fps Duration = 20000000 ; Analog Gain range min 1,000000, max 255,000000; Exposure Range min 68000, max 20000000;

GST_ARGUS: Running with following settings:
   Camera index = 1
   Camera mode  = 0
   Output Stream W = 1280 H = 800
   seconds to Run    = 0
   Frame Rate = 20,000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
H264: Profile = 66, Level = 0
...

In ISP mode I can also stop and restart one stream (VC0 or VC1) while another stream (VC1 or VC0) is active.

But the customer requirement is to use one virtual channel (VC0 or VC1) in VI mode and another virtual channel (VC1 or VC0) in ISP mode. And I observed various critical bugs with this usage scenario.
First problem is that the running ISP stream (VC0) stops if I start another stream (VC1) with v4l2-ctl.
How to reproduce:
First, start the VC0 stream by:

$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! "video/x-raw(memory:NVMM),width=1280,height=800,format=NV12,framerate=20/1" ! nvv4l2h264enc ! h264parse ! rtph264pay mtu=1400 ! udpsink host=192.168.10.30 port=5000

and while it is running, start the VC1 stream in VI mode:

v4l2-ctl -d /dev/video0 -c bypass_mode=0 --stream-mmap --stream-count=1000
<<<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.19 fps
...

This results in the broken ISP stream, camera data for VC0 is not recieved any more.
Below are the log files from dmesg, trace buffer and nvargus-daemon (journalctl -u):

dmesg-isp-stream-0-interrupted-by-v4l2-ctl-0.txt (76.6 KB)
trace-isp-stream-0-interrupted-by-v4l2-ctl-0.txt (184.6 KB)
nvargus-daemon-isp-stream-0-interrupted-by-v4l2-ctl-0.txt (159.6 KB)

Similar problem appears if I first start the VC1 stream (in VI mode), then start the VC0 stream (in ISP mode). Now both streams are running as expected, but if I now interrupt the VC1 stream, the ISP stream stops and nvargus-daemon reports errors.

Commands to reproduce:
First start VC1 stream:

$ v4l2-ctl -d /dev/video0 -c bypass_mode=0 --stream-mmap --stream-count=1000
<<<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.19 fps
...

While this is running, start the VC0 stream:

$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! "video/x-raw(memory:NVMM),width=1280,height=800,format=NV12,framerate=20/1" ! nvv4l2h264enc ! h264parse ! rtph264pay mtu=1400 ! udpsink host=192.168.10.30 port=5000

Now interrupt the VC1 stream with CTRL-C, and the ISP stream stops immediately.
Below are the log files from the latter case:

dmesg-isp-stream-0-interrupted-by-v4l2-ctl-stop.txt (122.3 KB)
trace-isp-stream-0-interrupted-by-v4l2-ctl-stop.txt (323.0 KB)
nvargus-daemon-isp-stream-0-interrupted-by-v4l2-ctl-stop.txt (769.3 KB)

Similar problem was already reported before here:
https://forums.developer.nvidia.com/t/missing-argus-support-for-low-frame-resolution-cameras/178537/18

I think Nvidia can reproduce these bugs on Jetson TX2 with GMSL + IMX390 cameras.

ShaneCCC · July 29, 2021, 1:18pm

Hi,
Had report internally for verifying it. Will update it once get any result.

agustg650e · July 29, 2021, 1:27pm

Hi,
many thanks!

agustg650e · July 29, 2021, 1:42pm

Another note: since the TX2 setup with GMSL + IMX390 uses a configuration with 2 CSI data lanes, I’ve reconfigured the camera, NVCSI and VI device tree nodes and the deserializer to use 2 CSI data lanes, too. But the reported issues were still reproducible with this new configuration, so this is not related to the different lane configuration.

ShaneCCC · July 30, 2021, 4:47am

@agustg650e
Yes, I think so the lane configure doesn’t matter with that.
Also I have the impression TX2 working well at early time when VC implement. No matter what will confirm it again with latest release to confirm it.

ShaneCCC · August 2, 2021, 2:54am

@agustg650e
Just verified on TX2 without problem for virtual channel use case.
You can saw nvgstcapture-1.0 was running in another console and v4l2-ctl run successfully.

raja@linux:~$ ps aux | grep -i nvgstcap
raja      8435 13.1  0.6 17500700 53312 pts/1  Sl+  22:52   0:02 nvgstcapture-1.0
raja      8516  0.0  0.0   7392   652 pts/0    S+   22:52   0:00 grep --color=auto -i nvgstcap
raja@linux:~$ v4l2-ctl --stream-mmap --stream-count=100 --set-ctrl bypass_mode=0 -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<

raja@linux:~$ media-ctl -p
Media controller API version 0.1.0

Media device information
------------------------
driver          tegra-vi4
model           NVIDIA Tegra Video Input Device
serial
bus info
hw revision     0x3
driver version  0.0.0

Device topology
- entity 1: 150c0000.nvcsi--2 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                <- "imx390 30-001b":0 [ENABLED]
        pad1: Source
                -> "vi-output, imx390 30-001b":0 [ENABLED]

- entity 4: imx390 30-001b (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:SRGGB12_1X12/1920x1080 field:none colorspace:srgb]
                -> "150c0000.nvcsi--2":0 [ENABLED]

- entity 6: vi-output, imx390 30-001b (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "150c0000.nvcsi--2":1 [ENABLED]

- entity 18: 150c0000.nvcsi--1 (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
        pad0: Sink
                <- "imx390 30-001c":0 [ENABLED]
        pad1: Source
                -> "vi-output, imx390 30-001c":0 [ENABLED]

- entity 21: imx390 30-001c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
        pad0: Source
                [fmt:SRGGB12_1X12/1920x1080 field:none colorspace:srgb]
                -> "150c0000.nvcsi--1":0 [ENABLED]

- entity 23: vi-output, imx390 30-001c (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
        pad0: Sink
                <- "150c0000.nvcsi--1":1 [ENABLED]

raja@linux:~$ media-ctl --print-dot
digraph board {
        rankdir=TB
        n00000001 [label="{{<port0> 0} | 150c0000.nvcsi--2\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
        n00000001:port1 -> n00000006
        n00000004 [label="{{} | imx390 30-001b\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
        n00000004:port0 -> n00000001:port0
        n00000006 [label="vi-output, imx390 30-001b\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
        n00000012 [label="{{<port0> 0} | 150c0000.nvcsi--1\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
        n00000012:port1 -> n00000017
        n00000015 [label="{{} | imx390 30-001c\n/dev/v4l-subdev3 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
        n00000015:port0 -> n00000012:port0
        n00000017 [label="vi-output, imx390 30-001c\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
}

agustg650e · August 3, 2021, 7:43am

Did you test it remotely?
How do you know that the nvgstcapture-1.0 is still capturing images successfully after v4l2-ctl command finished?

ShaneCCC · August 3, 2021, 9:00am

Right, I will check it if can capture jpg image by nvgstcapture-1.0 to confirm it.

ShaneCCC · August 3, 2021, 9:05am

Looks like run the v4l2-ctl first then run nvgstcapture-1.0 without problem.
But get failed if run nvgstcapture-1.0 then run v4l2-ctl.

agustg650e · August 3, 2021, 9:54am

Another (and easier way to reproduce it) when testing remotely is to use gst-launch-1.0 with fakesink.
Below my commands:

First run

$ gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=0 ! "video/x-raw(memory:NVMM),width=1280,height=800,format=NV12,framerate=20/1" ! fakesink silent=0
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: stream-start (10254), GstEventStreamStart, stream-id=(string)cb779ee2a12d84c4223ee690bc5af0be, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)1;) 0x55a9736d10
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1280, height=(int)800, format=(string)NV12, framerate=(fraction)20/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1280, height=(int)800, format=(string)NV12, framerate=(fraction)20/1
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: caps (12814), GstEventCaps, caps=(GstCaps)"video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1280\,\ height\=\(int\)800\,\ format\=\(string\)NV12\,\ framerate\=\(fraction\)20/1";) 0x55a9736df0
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1280, height=(int)800, format=(string)NV12, framerate=(fraction)20/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1280, height=(int)800, format=(string)NV12, framerate=(fraction)20/1
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1280 x 800 FR = 20,000000 fps Duration = 50000000 ; Analog Gain range min 1,000000, max 15,937500; Exposure Range min 9000, max 10000000;

GST_ARGUS: 1280 x 800 FR = 50,000000 fps Duration = 20000000 ; Analog Gain range min 1,000000, max 255,000000; Exposure Range min 68000, max 20000000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 1280 H = 800 
   seconds to Run    = 0 
   Frame Rate = 20,000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: segment (17934), GstEventSegment, segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0, offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0, duration=(guint64)18446744073709551615;";) 0x55a9736e60
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = preroll   ******* 
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.535123872, pts: 0:00:00.535123872, duration: none, offset: -1, offset_end: -1, flags: 00000040 discont , meta: none) 0x7f9c00a030
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.584433824, pts: 0:00:00.584433824, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a140
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.634303424, pts: 0:00:00.634303424, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a250
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.683244896, pts: 0:00:00.683244896, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a360
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.733331712, pts: 0:00:00.733331712, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a470
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.782979584, pts: 0:00:00.782979584, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a580
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.832685568, pts: 0:00:00.832685568, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a030
...

Now start v4l2-ctl in another console:

$ v4l2-ctl --stream-mmap --stream-count=200 --set-ctrl bypass_mode=0 -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<<<< 20.14 fps
<<<<<<<<<<<<<<<<<<<< 20.11 fps
<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<<<< 20.14 fps
<<<<<<<<<<<<<<<<<<<< 20.12 fps
<<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<

As soon as v4l2-ctl is started, fakesink stops outputting the messages (because frame reception in argus failed).

Now cancel the above gst-launch-1.0 pipeline with CTRL-C, and then restart the nvargust-daemon to test another problematic case:

sudo systemctl restart nvargus-daemon

The another test to reproduce is to start v4l2-ctl first, but use some more buffers to capture before finishing (so that we can start gst-launch-1.0 with fakesink before v4l2-ctl finished). When v4l2-ctl finishes,
the gst-launch-1.0 pipeline stops streaming:

$ v4l2-ctl --stream-mmap --stream-count=200 --set-ctrl bypass_mode=0 -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<<<< 20.14 fps
<<<<<<<<<<<<<<<<<<<< 20.11 fps
<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<<<< 20.14 fps
<<<<<<<<<<<<<<<<<<<< 20.12 fps
<<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<

Argus stream start just after v4l2-ctl launch:

$ gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=0 ! "video/x-raw(memory:NVMM),width=1280,height=800,format=NV12,framerate=20/1" ! fakesink silent=0
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: stream-start (10254), GstEventStreamStart, stream-id=(string)fd0112fbfd2fbe45734ddc1aaac18128, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)1;) 0x5584416510
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1280, height=(int)800, format=(string)NV12, framerate=(fraction)20/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1280, height=(int)800, format=(string)NV12, framerate=(fraction)20/1
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: caps (12814), GstEventCaps, caps=(GstCaps)"video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1280\,\ height\=\(int\)800\,\ format\=\(string\)NV12\,\ framerate\=\(fraction\)20/1";) 0x55844165f0
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1280, height=(int)800, format=(string)NV12, framerate=(fraction)20/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1280, height=(int)800, format=(string)NV12, framerate=(fraction)20/1
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1280 x 800 FR = 20,000000 fps Duration = 50000000 ; Analog Gain range min 1,000000, max 15,937500; Exposure Range min 9000, max 10000000;

GST_ARGUS: 1280 x 800 FR = 50,000000 fps Duration = 20000000 ; Analog Gain range min 1,000000, max 255,000000; Exposure Range min 68000, max 20000000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 1280 H = 800 
   seconds to Run    = 0 
   Frame Rate = 20,000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event   ******* (fakesink0:sink) E (type: segment (17934), GstEventSegment, segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0, offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0, duration=(guint64)18446744073709551615;";) 0x5584416660
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = preroll   ******* 
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.506018336, pts: 0:00:00.506018336, duration: none, offset: -1, offset_end: -1, flags: 00000040 discont , meta: none) 0x7f9c00a040
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.554866080, pts: 0:00:00.554866080, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a150
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.604582912, pts: 0:00:00.604582912, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a260
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.654422304, pts: 0:00:00.654422304, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a370
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.703959904, pts: 0:00:00.703959904, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a480
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain   ******* (fakesink0:sink) (1008 bytes, dts: 0:00:00.753890528, pts: 0:00:00.753890528, duration: none, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f9c00a590
...

ShaneCCC · August 3, 2021, 12:12pm

You have the same result for run v4l2-ctl first?

agustg650e · August 3, 2021, 2:00pm

Not really the same result. For the test case with running v4l2-ctl first, the result is that that the nvgstcapture-1.0 can be started, but after taking pictures and restarting nvgstcapture-1.0, sometimes taking pictures does not work any more. And when stopping nvgstcapture-1.0, there is a timeout error reported in the dmesg log (for the running v4l2-ctl):

[ 5340.111381] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[ 5340.117751] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel

And v4l2-ctl frame rate has temporary drops:

$ v4l2-ctl --stream-mmap --stream-count=20000 --set-ctrl bypass_mode=0 -d /dev/video0
<<<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.19 fps
<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<<<< 20.14 fps
<<<<<<<<<<<<<<<<<<<< 20.11 fps
<<<<<<<<<<<<<<<<<<<< 20.13 fps
<<<<<<<<<<<<<<<<<<<< 20.14 fps
<<<<<<<<<<<<<<< 19.50 fps
<<<<<<<<<<<<<<<<<<<<< 19.57 fps
<<<<<<<<<<<<<<<<<<<< 19.62 fps
<<<<<<<<<<<<<<<<<<<< 19.67 fps
<<<<<<<<<<<<<<<<<<<< 19.71 fps
<<^C

It is also reproducible with below command started repeatedly, while v4l2-ctl is running:

$ gst-launch-1.0 -v nvarguscamerasrc num-buffers=5 sensor-id=0 sensor-mode=0 ! "video/x-raw(memory:NVMM),width=1280,height=800,format=NV12,framerate=20/1" ! fakesink silent=0

ShaneCCC · August 4, 2021, 3:25am

OK, I have reported to developer to fix this problem.
Will provide solution once get any result.

agustg650e · September 23, 2021, 9:37am

Any update on this topic? Will these bugs be fixed in next L4T/JetPack release?

ShaneCCC · February 9, 2022, 1:05pm

Sorry to tell this feature are pending due to resource issue.
I would suggest to use Xavier/Xavier NX for this kind of use case if have schedule timing problem.

system · March 9, 2022, 12:58am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Failing to get input from CSI camera (IMX219-83) Jetson Nano camera , ubuntu , gstreamer	29	1427	September 25, 2023
Problems with nvarguscamerasrc using the last imx565 mode Jetson AGX Xavier camera , gstreamer	43	344	July 11, 2024
Problems on porting ov9281 csi camera on our carrier board Jetson TX2 camera	51	3471	April 21, 2022
Gstreamer reports Raspberry Pi camera streaming at 120fps when in reality it is only 60fps Jetson Nano camera , gstreamer	53	3775	July 19, 2022
Argus stability on Orin NX Jetson Orin NX camera , gstreamer	28	1315	August 28, 2023
Inconsistent gStreamer with IMX219/477 Jetson Xavier NX gstreamer	5	1434	October 31, 2022
Unable to stream video from IMX296 sensor on Orin NX Jetson Orin NX camera , gstreamer , sensor	26	1302	January 3, 2024
CSI Camera imx219-77 never pops up video window Jetson Nano camera	7	1203	October 15, 2021
Timeout on /dev/video3 capture Jetson AGX Orin camera	9	483	February 1, 2024
120 fps mode support removed for imx219 sensor Jetson Nano gstreamer	17	5988	October 15, 2021

TX2 VI mode and ISP mode bugs when using CSI2 virtual channels

Related topics