No captured data from v4l2 driver (TC358743)

If you want to display the captured video live on the Jetson Nano (currently displays a green image!) you can run the command:

gst-launch-1.0 -vvvvv v4l2src ! nvvidconv ! nvegltransform ! nveglglessink -e

Cheers

Phil

1 Like

@phil1 @robert.j.h.chandler
just googled “gst-launch green screen” and found a lot of topics, one saying :

‘green’ color indicates uninitialized memory in YUV formats. This could eg. be the nvvidconv not filling all pars of the buffer.

https://www.google.com/search?q=gst-launch+green+screen&oq=gst-launch+green+screen&aqs=chrome..69i57j0.4841j0j4&sourceid=chrome&ie=UTF-8

https://stackoverflow.com/questions/43816676/getting-green-screen-in-video-streaming-through-janus-and-gstreamer

1 Like

so I guess the shared memory which VI module was about to write is empty ? that could be the reason of green screen

I also wonder if we’ve got the bus width/port index correct in the DT

The ‘Port Index’ section in the Sensor Programming Guide would suggest that the bus width of CSI-B is only 2 where in the DT I specified a bus width of 4. I’m wondering if the connector on the dev kit is connected to CSI-A and CSI-B or just CSI-B?

From the schematic it seems like theFPC connector is for CSI0 with 4 lanes, but it’s not clear what CSI0 means in terms of the CSI-A, CSI-B, etc.

I altered DTSI file :

#include <dt-bindings/media/camera.h>
#include <dt-bindings/platform/t210/t210.h>
#include <dt-bindings/gpio/gpio.h>

/ {
host1x {

    vi_base: vi {
        num-channels = <1>;  // Change 4->2
        ports {
            #address-cells = <1>;
            #size-cells = <0>;

            vi_port1: port@0 {
                status = "okay";
                reg = <0>;
                tc358743_vi_in1: endpoint {
                    status = "okay";
                    port-index = <0>;  /* CSI-B */
                    bus-width = <2>; /* Use CSI-B only */
                    remote-endpoint = <&tc358743_csi_out0>;
                };
            };
        };
    };


    csi_base: nvcsi {
        num-channels = <1>;
        #address-cells = <1>;
        #size-cells = <0>;

        channel@0 {
            reg = <0>;
            ports {
                #address-cells = <1>;
                #size-cells = <0>;
                port@0 {
                    status = "okay";
                    reg = <0>;
                    tc358743_csi_in0: endpoint@0 {
                        status = "okay";
                        port-index = <0>;
                        bus-width = <2>;
                        remote-endpoint = <&tc358743_out1>;
                    };
                };
                port@1 {
                    reg = <1>;
                    status = "okay";
                    tc358743_csi_out0: endpoint@1 {
                        status = "okay";
                        remote-endpoint = <&tc358743_vi_in1>;
                    };
                };
            };
        };
    };

    i2c@546c0000 {  /* I2C_PM, "adapter" 6 */
        status = "okay";
        #address-cells = <1>;
        #size-cells = <0>;
        tc358743@0f {
            status = "okay";
            compatible = "tc358743";
            reg = <0x0f>; /* shifted by 2 */
            mclk = "cam_mclk1";
            reset-gpios = <&gpio 149 0>;
            refclk_hz = <27000000>;  // refclk_hz -> regclk

	        interrupt-parent = <&gpio>;
	        interrupts = <TEGRA_GPIO(E, 6) GPIO_ACTIVE_HIGH>;

            /* Physical dimensions of sensor */
            physical_w = "4.713";
            physical_h = "3.494";
            /* Sensor Model */
            sensor_model ="tc358743";
            
   

            ddc5v_delay = <2>;
            enable_hdcp = "false";
            lineinitcnt = <0xe80>;
            lptxtimecnt = <0x003>;
            tclk_headercnt = <0x1403>;
            tclk_trailcnt = <0x00>;
            ths_headercnt = <0x0103>;
            twakeup = <0x4882>;
            tclk_postcnt = <0x008>;
            ths_trailcnt = <0x02>;
            hstxvregcnt = <0>;

            ports {
                #address-cells = <1>;
                #size-cells = <0>;

                port@0 {
                    reg = <0>;
                    tc358743_out1: endpoint {
                        port-index = <0>; /* CSI B */
                        bus-width = <2>; /* Use CSI-B only */
                        data-lanes = <1 2>;
                        clock-lanes = <0>;
                        clock-noncontinuous;
                        link-frequencies = /bits/ 64 <297000000>;
                        remote-endpoint = <&tc358743_csi_in0>;
                    };
                };
            };
        };
    };
};

tegra-camera-platform {
    status = "okay";
	compatible = "nvidia, tegra-camera-platform";
	num_csi_lanes = <2>;  // Changed 2 -> 4
	max_lane_speed = <1500000>;
	min_bits_per_pixel = <10>;  // Changed 16 -> 10
	vi_peak_byte_per_pixel = <2>;
	vi_bw_margin_pct = <25>;
	max_pixel_rate = <750000>;
	isp_peak_byte_per_pixel = <5>;
	isp_bw_margin_pct = <25>;

    	/**
	* The general guideline for naming badge_info contains 3 parts, and is as follows,
	* The first part is the camera_board_id for the module; if the module is in a FFD
	* platform, then use the platform name for this part.
	* The second part contains the position of the module, ex. “rear” or “front”.
	* The third part contains the last 6 characters of a part number which is found
	* in the module's specsheet from the vender.
	*/
	modules {

	    module1 {
	        status = "okay";
	        badge = "tc358743_top_i2c6_b";
	        position = "front";
	        orientation = "1";
	        drivernode0 {
	            status = "okay";
	            /* Declare PCL support driver (classically known as guid)  */
	            pcl_id = "v4l2_sensor";
	            /* Driver's v4l2 device name */
	            devname = "tc358743 6-000f";
	            /* Declare the device-tree hierarchy to driver instance */
	            proc-device-tree = "/proc/device-tree/host1x/i2c@546c0000/tc358743@0f";
	        };
	    };
        
	};
};
};

it works normally, except this time gst-launch shows a frozen-transparent window instead of a green window:

output of “media-ctl -p -d /dev/media0” :

root@t-desktop:~/i2c# media-ctl -p -d /dev/media0 
Media controller API version 0.1.0

Media device information
------------------------
driver          vi
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  0.0.0

Device topology
- entity 1: nvcsi--1 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "tc358743 6-000f":0 [ENABLED]
	pad1: Source
		-> "vi-output, tc358743 6-000f":0 [ENABLED]

- entity 4: tc358743 6-000f (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
	pad0: Source
		[fmt:UYVY8_1X16/1920x1080 field:none colorspace:smpte170m]
		[dv.caps:BT.656/1120 min:1x1@0 max:10000x10000@165000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
		[dv.detect:BT.656/1120 1920x1080p50 (2640x1125) stds: flags:]
		[dv.current:BT.656/1120 1920x1080p50 (2640x1125) stds:CEA-861 flags:CE-video]
		-> "nvcsi--1":0 [ENABLED]

- entity 6: vi-output, tc358743 6-000f (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
	pad0: Sink
		<- "nvcsi--1":1 [ENABLED]

Hi Robert,

Looking at your latest source, you have commented out the refclk section within the probe function. I know that previous people that have tried to get it working discovered that the tc358743 requires a 27Mhz clock but the maximum that the Nano can provide is 24Mhz. There was no solution other than to use a different clock. I suspect that without a functional reference clock, it will be unable to stream anyway?

Cheers

Phil

I’m sure there will be a usable clock source , I’ve seen some other people built a working prototype without physical modifications to dev board :)

Yep as @electrixoul says it’s possible to get it working without physical changes to the board. I don’t think the Nano needs to provide a clock source because the TC358743 breakout boards have one on board. I can see a 27Mhz clock on the B101 board that I’ve got.

I think we’re getting close, just need to sort out the final device tree bits I think.

probably a similar issue :

The bus-width is still 2 in some places in your DT, I think to stream 1080p (especially at higher framerates) you need more than 2 lanes

1 Like

I’m sure it is possible and I have seen some mention of using a different clock source (extperiph1) instead of the default. I am just playing around with that now and will let you know if I discover anything.

2 Likes

Using your DT (with bus-width set to 4) no successful frames are being received (if you look at the test.ts file it’s zero bytes).

Looking at the kernel logs (dmesg) I get the following:

[  242.106181] tc358743 6-000f: -----DVI-D status-----
[  242.106184] tc358743 6-000f: HDCP encrypted content: no
[  242.106188] tc358743 6-000f: Input color space: RGB full range
[  242.116779] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 0
[  242.136735] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 1
[  242.156805] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 2
[  242.176666] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 3
[  242.197035] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 4
[  242.217181] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 5
[  242.237234] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 6
[  242.257246] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 7

Time to investigate that error…

1 Like

actually I think bus-width=2 should be enough because official IMX219 camera driver on nano is using “bus-width=2” , also I tested IMX219 camera on jetson Nano , 1080p @ 60fps was never an issue :-)

@electrixoul, @phil1 I GOT IT WORKING!!! Disclaimer: currently only tested 720p, but it works! Check out the gsit for the updated code and device tree.

gst-launch-1.0 v4l2src device=/dev/video0 ! 'video/x-raw,format=UYVY,width=1280,height=720,framerate=60/1' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420,width=1920,height=1080' ! nvoverlaysink

P.S. @electrixoul The reason I don’t think it’s enough is that when you do v4l2-ctl --log-status it shows ‘lanes needed’ and for 720p@50 it shows 2 and for 1080p@50 it shows 3

3 Likes

WOOOOOW !

you are good !

Superb! I will try it now.

Cheers

Phil

Thanks but honestly I’ve been trying to get this working for the last 6 weeks and learnt a ton in the process. Without your help on the device trees and commenting out the imx lines I think I might have given up! I’m also mostly using other peoples code from around the internet and trying to piece it together.

Just got to get 1080p working now, I tried and I doesn’t work with the current setup, most likely a device tree issue but possibly something in the driver.

1 Like

It seems like I also currently have to call

v4l2-ctl --device /dev/video0 --stream-mmap --set-fmt-video=width=1280,height=720,pixelformat=UYVY

To get the format configured correctly before the streaming works, this is probably something we can fix in the driver by getting it to set the format before starting streaming.

1 Like