Tk1 supports gstreamer or not

We added a bayer camera on Tk1 R21.5 and the camera driver is modified from Tk1 imx135_v4l2.c.
We failed to use gstreamer but could capture a frame by using yavta.
Is there any we missed?

Belwo are the commands we used and the log.
./yavta /dev/video0 -c1 -n1 -s1920x1080 -fSRGGB10 -Fimx1x5.raw
gst-launch-1.0 nvcamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12’ ! nvoverlaysink -ev

ubuntu@tegra-ubuntu:~$ ./yavta /dev/video0 -c1 -n1 -s1920x1080 -fSRGGB10 -Fimx1x5.raw
Device /dev/video0 opened.
Device vi' on ’ is a video capture device.
Video format set: SRGGB10 (30314752) 1920x1080 (stride 3840) buffer size 4147200
Video format: SRGGB10 (30314752) 1920x1080 (stride 3840) buffer size 4147200
1 buffers requested.
length: 4147200 offset: 0
Buffer 0 mapped at address 0xb69de000.
0 (0) [E] 0 4147200 bytes 1491904018.880566 500.980152 -0.002 fps
Captured 1 frames in 1.967854 seconds (0.508168 fps, 2107472.440365 B/s).
1 buffers released.

ubuntu@tegra-ubuntu:~$ gst-launch-1.0 nvcamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12’ ! nvoverlaysink -ev
Setting pipeline to PAUSED …
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingNvCamHwHalDeviceRead: 18 - -1
NvCamHwHalDeviceRead: 19 - -1
NvCamHwHalDeviceRead: 20 - -1
NvCamHwHalDeviceRead: 22 - -1
NvCamHwHalDeviceRead: 23 - -1
NvCamHwHalDeviceInstall: PCLLK_IOCTL_DEV_REG fail. -1
ImagerDeviceDetect: Failed to register new device
IMX135 **** Can not open camera device: No such file or directory
NvOdmImagerOpenExpanded 462: Sensor ERR
NvOdmImagerOpenExpanded FAILED!
camera_open failed
ERROR: Pipeline doesn’t want to pause.
Setting pipeline to NULL …
Freeing pipeline …

error message on host:
i2c i2c-2: Failed to register i2c client pcl_IMX1x5 at 0x1a (-16)
pcl-generic pcl-generic: camera_new_device cannot allocate client: pcl_IMX1x5 bus 2, 1a

Thank you,

Hi linuxsky
Sorry to let you know. There’s no way to use nvcamerasrc on TK1 due to the software architecture is different with TX1. For TK1 you only can use v4l2src and bayer2rgb element for your project.

Hi linuxsky,

The command should like this:
gst-launch-1.0 v4l2src device=“/dev/video0” ! ‘video/x-bayer,format=bggr,width=1280,height=720’ ! bayer2rgb ! videoconvert ! xvimagesink sync=false

Thanks

Dear ShaneCCC and KayCCC,

Thank you for your prompt support.

We encountered an error, as we tested the v4l2src command you provided. (message below.)
We will re-check our driver and appreciate your any suggestions.

Plus, one more question.
Does it use Tk1 ISP as using v4l2src and bayer2rgb element.

ubuntu@tegra-ubuntu:~$ gst-launch-1.0 v4l2src device=“/dev/video0” ! ‘video/x-bayer,format=bggr,width=1280,height=720’ ! bayer2rgb ! videoconvert ! xvimagesink sync=false
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2865): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:00.735842046
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
ubuntu@tegra-ubuntu:~$

Thank you,

It might be a permission issue for accessing /dev/video0.
Have you tried with sudo ?

Hi linuxsky
Would you remove the format=bggr to verify.

Thank you Hobey Patouceul and ShaneCCC,

Sorry. Both sudo and remove foramt=bggr got the same error.

ubuntu@tegra-ubuntu:~$ gst-launch-1.0 v4l2src device=“/dev/video0” ! ‘video/x-bayer,width=1920,height=1080’ ! bayer2rgb ! videoconvert ! xvimagesink sync=false
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2865): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:00.731233233
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
ubuntu@tegra-ubuntu:~$ gst-launch-1.0 v4l2src device=“/dev/video0” ! ‘video/x-bayer,width=1920,height=1080’ ! bayer2rgb ! videoconvert ! xvimagesink sync=false
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2865): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:00.751204909
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
ubuntu@tegra-ubuntu:~$ sudo gst-launch-1.0 v4l2src device=“/dev/video0” ! ‘video/x-bayer,width=1920,height=1080’ ! bayer2rgb ! videoconvert ! xvimagesink sync=false
[sudo] password for ubuntu:
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2865): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:00.733260232
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
ubuntu@tegra-ubuntu:~$

Could you try below command to see narrow down which element get problem.

gst-launch-1.0 v4l2src device="/dev/video0" ! 'video/x-bayer,width=1920,height=1080' ! fakesink
gst-launch-1.0 v4l2src device="/dev/video0" ! 'video/x-bayer,width=1920,height=1080' ! bayer2rgb ! fakesink
gst-launch-1.0 v4l2src device="/dev/video0" ! 'video/x-bayer,width=1920,height=1080' ! bayer2rgb ! videoconvert ! fakesink

Hi linuxsky,

Have you tried our suggestion?
Any update?

Thanks

Dear Kayccc,

Thank you for asking.

We are still struggling to figure out our issue is caused by sensor driver or sensor HW.
Due to we are first bring up camera sensor on Tk1 and the sensor module is new too.

We could get a frame with format=bggr removed after the sensor driver was modified.
But we still failed to get multi frames.

Here is the error message.
[ 2755.558469] vi vi.0: CSI_A syncpt timeout, syncpt = 4, err = -11
[ 2755.573603] TEGRA_CSI_CSI_CIL_A_STATUS 0x00000010
[ 2755.586941] TEGRA_CSI_CSI_CILA_STATUS 0x00040041
[ 2755.598309] TEGRA_CSI_CSI_CIL_B_STATUS 0x00000000
[ 2755.603449] TEGRA_CSI_CSI_CIL_C_STATUS 0x00000000
[ 2755.614055] TEGRA_CSI_CSI_CIL_D_STATUS 0x00000000
[ 2755.624539] TEGRA_CSI_CSI_CIL_E_STATUS 0x00000000
[ 2755.636316] TEGRA_CSI_CSI_PIXEL_PARSER_A_STATUS 0x00000000
[ 2755.649229] TEGRA_CSI_CSI_PIXEL_PARSER_B_STATUS 0x00000000
[ 2755.658182] TEGRA_VI_CSI_0_ERROR_STATUS 0x0000000a
[ 2755.672339] TEGRA_VI_CSI_1_ERROR_STATUS 0x00000000
[ 2756.394890] mc-err: [mcerr] (vi) csw_viw: EMEM decode error on PDE or PTE entry
[ 2756.405547] mc-err: [mcerr] status = 0x60053072; addr = 0x8041e4c0
[ 2756.411719] mc-err: [mcerr] secure: no, access-type: write, SMMU fault: nr-nw-s
fps, 10246614.151484 B/s).

The thing is we actually got error message below as even getting a frame by yavta command.
But we could get a correct frame/picture.
Although the test pattern we got does not completely correct.

[ 272.278243] vi vi.0: CSI_A syncpt timeout, syncpt = 6, err = -11
[ 272.298839] TEGRA_CSI_CSI_CIL_A_STATUS 0x00000010
[ 272.310671] TEGRA_CSI_CSI_CILA_STATUS 0x00040041
[ 272.323474] TEGRA_CSI_CSI_CIL_B_STATUS 0x00000000
[ 272.330621] TEGRA_CSI_CSI_CIL_C_STATUS 0x00000000
[ 272.345497] TEGRA_CSI_CSI_CIL_D_STATUS 0x00000000
[ 272.351040] TEGRA_CSI_CSI_CIL_E_STATUS 0x00000000
[ 272.363857] TEGRA_CSI_CSI_PIXEL_PARSER_A_STATUS 0x00000000
[ 272.379681] TEGRA_CSI_CSI_PIXEL_PARSER_B_STATUS 0x00000000
[ 272.388606] TEGRA_VI_CSI_0_ERROR_STATUS 0x0000000a
[ 272.402827] TEGRA_VI_CSI_1_ERROR_STATUS 0x00000000

We will check the sensor driver again.
Thank you for any suggestions.

Thanks,

They all report that there is Control Error.
Control error is set when proper LP → HS or HS → LP
transitioning is not detected.
This may or may not be a real issue.
For any real issue, You should check the MIPI bus to see if sensor is transitioning
through this phase properly.
Usually such issue will manifest itself in the form of frame loss.

But if you do not see any missing frame, then they could be set due
to software sequencing.
When the CIL (CSI Phy layer) is enabled, sometimes the
MIPI bus could be in LP00 state, Then the internal logic
treats this as error, since it expects the MIPI signals to be in logic 11.
You can resolve this error by making sure the MIPI bus is in the LP11 state,
before you enable the CIL. In other words turn on the Receiver before you turn
on the transmitter.

Dear ShaneCCC,

Thank you for your support.
Attached some waves of the camera hsync (c1)/ mipi lane0 clock (c2)/lane0 data (c3).

We still cannot get multi frames.
one frame — ok & image ok
two frames — ok & first image ok , second image is black
three frames — ok, & first image ok , second/third images are black
four frames — hang up

Below is the message of getting 4 frames by yavta.
[ 170.621916] vi vi.0: CSI_A syncpt timeout, syncpt = 8, err = -11
[ 170.638036] TEGRA_CSI_CSI_CIL_A_STATUS 0x00000010
[ 170.666404] TEGRA_CSI_CSI_CILA_STATUS 0x00040041
[ 170.676479] TEGRA_CSI_CSI_CIL_B_STATUS 0x00000000
[ 170.686792] TEGRA_CSI_CSI_CIL_C_STATUS 0x00000000
[ 170.705316] TEGRA_CSI_CSI_CIL_D_STATUS 0x00000000
[ 170.716384] TEGRA_CSI_CSI_CIL_E_STATUS 0x00000000
[ 170.726751] TEGRA_CSI_CSI_PIXEL_PARSER_A_STATUS 0x00000000
[ 170.736516] TEGRA_CSI_CSI_PIXEL_PARSER_B_STATUS 0x00000000
[ 170.742999] TEGRA_VI_CSI_0_ERROR_STATUS 0x0000000a
[ 170.748244] TEGRA_VI_CSI_1_ERROR_STATUS 0x00000000
[ 170.970915] vi vi.0: CSI_A syncpt timeout, syncpt = 10, err = -11
[ 170.987382] TEGRA_CSI_CSI_CIL_A_STATUS 0x00000010
[ 171.004022] TEGRA_CSI_CSI_CILA_STATUS 0x00040041
[ 171.023984] TEGRA_CSI_CSI_CIL_B_STATUS 0x00000000
[ 171.035599] TEGRA_CSI_CSI_CIL_C_STATUS 0x00000000
[ 171.049471] TEGRA_CSI_CSI_CIL_D_STATUS 0x00000000
[ 171.061046] TEGRA_CSI_CSI_CIL_E_STATUS 0x00000000
[ 171.072994] TEGRA_CSI_CSI_PIXEL_PARSER_A_STATUS 0x00000000
[ 171.079950] TEGRA_CSI_CSI_PIXEL_PARSER_B_STATUS 0x00000000
[ 171.085516] TEGRA_VI_CSI_0_ERROR_STATUS 0x0000000a
[ 171.090493] TEGRA_VI_CSI_1_ERROR_STATUS 0x00000000
[ 171.315470] vi vi.0: CSI_A syncpt timeout, syncpt = 12, err = -11
[ 171.345982] TEGRA_CSI_CSI_CIL_A_STATUS 0x00000010
[ 171.363440] TEGRA_CSI_CSI_CILA_STATUS 0x00040041
[ 171.369117] TEGRA_CSI_CSI_CIL_B_STATUS 0x00000000
[ 171.374202] TEGRA_CSI_CSI_CIL_C_STATUS 0x00000000
[ 171.379629] TEGRA_CSI_CSI_CIL_D_STATUS 0x00000000
[ 171.384686] TEGRA_CSI_CSI_CIL_E_STATUS 0x00000000
[ 171.390892] TEGRA_CSI_CSI_PIXEL_PARSER_A_STATUS 0x00000000
[ 171.396747] TEGRA_CSI_CSI_PIXEL_PARSER_B_STATUS 0x00000000
[ 171.402720] TEGRA_VI_CSI_0_ERROR_STATUS 0x0000000a
[ 171.407878] TEGRA_VI_CSI_1_ERROR_STATUS 0x00000000
[ 171.630868] vi vi.0: CSI_A syncpt timeout, syncpt = 14, err = -11
[ 171.729766] MSELECT error detected! status=0x1

Thank you for any suggetsions.

Hi linuxsky

  1. The attachments are not unavailable, could you zip it and upload again.
  2. Could you try the discontinuous mode.

Dear ShaneCCC,

I attached the zip file first.
Sorry. I could not find a way to enable discontinuous mode. (neither Sony data sheet nor driver examples.)
I will try to figure out it later.

Thank you,
VR_imx_wave.tar (273 KB)

Moving the discussion thread to Jetson TK1 discussion board as this is Jetson TK1 issue.

Hi linuxsky,

Have you figured out how to enable discontinuous mode?
Could you make sure the mipi timing is correct?

Any further information can be shared?

Thanks

Hi Kayccc,

Sorry. We still don’t know how to do it.
we appreciate any sample or information of enabling discontinuous mode.
We will try to get a proper measure equipment to check the mipi timing and update it.
Thank you for your support again.

Hi linuxsky

Have you managed to get the equipment to measure the mipi timing?
Any result can be shared?

Thanks

Hi Kayccc,

Sorry. We have not got the equipment yet.
I’m afraid there is no any update result.
Also, we are considering to use a workable camera module on TK1 for test first.

Thank you for your support.