Camera outputs to 1 buffer only

3dimi · April 27, 2018, 7:55am

Hi,

We are trying to get raw frames from a camera using v4l2 but it looks like it is only writing into 1 v4l2 buffer (buffer.index = 0). Other buffers have null/black images. There is also an image tear in the captured frames. Only the first frame is consistently complete. The others that are not null usually has the bottom part of the frame coming from a previous time stamp (buffer[0].frame[1] has lines that exactly the same as buffer[0].frame[0]).

argus_camera and nvgstcapture are displaying good video streams. However, we cannot use these because we need to process raw frames from the camera.

qv4l2 (v4l2 test bench GUI) seems to be showing the same problem because the output video stream has alternating black and good frames.

Output from image sensor is parallel then goes through serializer to CSI input of TX1. We are not quite sure whether it is an issue with serializer HW or the driver.

Can anyone suggest any other things to check or possible solutions to this issue?

ShaneCCC · April 27, 2018, 8:11am

Please try the attached patch.

0001-drivers-camera-fix-FE-syncpt-wait.patch.txt (3.02 KB)
0002-media-tegra-fix-missing-timestamp-for-vi4.patch.txt (3.52 KB)

3dimi · April 27, 2018, 8:17am

Hi Shane,

Thanks for your prompt reply.

Are these patches compatible with R28.1? We haven’t updated to the latest JetPack yet.

3dimi · May 3, 2018, 6:27am

Hi Shane,

I’ve already update to L4T R28.2 and applied the patch you provided. I also translated the driver for the camera that we are using (sensor not yet included in the drivers available in the release). I was able to compile the kernel and DTB successfully.

However, video0 is no longer detected. It was working in the R28.1 version.

Can you help check what is the problem with the R28.2 version of our camera driver? I just translated the driver based on the Camera Sensor Drivers Porting Guide (28.1 to 28.2). However, I did not use the sensor_mode_properties, and just used the frame length registry address when setting the frame length, unlike the other camera drivers in the R28.2 release.
Can you also provide the R28.1 version of the patch files that you uploaded here? I just want to make sure that this will indeed fix the 1 buffer problem that we are having.

Thanks.

ShaneCCC · May 3, 2018, 6:55am

28.1 should have the problem need those patch to fixed. Your problem may not the same.
Your problem may be the alignment problem. Does your resolution 256 alignment? If not try to modify the below to 1 to try.

./camera/vi/core.h:#define TEGRA_STRIDE_ALIGNMENT       256

3dimi · May 3, 2018, 8:52am

Hi Shane,

It is currently set at 256 stride alignment. I’m compiling now with your suggested setting to TEGRA_STRIDE_ALIGNMENT 1 for R28.1.

By the way, I was able to fix the R28.2 and detect video0. However, the output frames are now all black.

ShaneCCC · May 3, 2018, 9:07am

How many frames did you dump all of them are black? Does the hexdump all the same value?

3dimi · May 3, 2018, 9:22am

~50 frames but not all of them are black.

I might also have a problem with the power rail in the driver. The camera only successfully turns on if the deserializer board that we are using has a floating ground. But it looks like the buffers are being used now (not just the first one). I just need to figure out this new issue.

3dimi · May 4, 2018, 1:53am

Hi Shane,

I reflashed the TX1 board and was now able to power on the camera with no errors.

It is also writing to more buffers now, instead of only writing to buffer.index = 0. However, it is still intermittent. Some buffers are still empty/black and almost always the first 4 frames that I dequeue is completely black.

The time between each dequeue also doesn’t make sense. It is faster than the expected frame rate.

I also still get the image tear where the bottom lines of the frame are from a previous timestamp or previous frame.

Is it because the buffers are dequeued even if they are not yet done (on the outgoing queue)?

ShaneCCC · May 4, 2018, 2:22am

Is it YUV sensor?

3dimi · May 4, 2018, 2:29am

No, it is an RGB-IR sensor that is why we need to get the raw image from the sensor.

3dimi · May 4, 2018, 7:05am

Hi Shane,

The driver might not be successfully writing to the memory buffer.
The saved frames are sometimes identical if they came from the same buffer index.
The buffer flags are also mostly errors (0x2045).
The kernel log also indicates errors in the vi and syncpoint time out:

May  4 06:54:35 tegra-ubuntu kernel: [  797.510958] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  797.533660] vi 54080000.vi: tegra_channel_error_status:error 20022 frame 5
May  4 06:54:35 tegra-ubuntu kernel: [  797.540912] vi 54080000.vi: tegra_channel_error_status:error 20022 frame 6
May  4 06:54:35 tegra-ubuntu kernel: [  797.647168] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  797.779628] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  797.911282] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  798.043267] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  798.179801] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  798.311191] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.443383] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.579441] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.711398] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.843513] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.979597] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  799.111556] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  799.243996] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  799.375920] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.511650] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.532287] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 82
May  4 06:54:37 tegra-ubuntu kernel: [  799.539420] vi 54080000.vi: tegra_channel_error_status:error 24022 frame 83
May  4 06:54:37 tegra-ubuntu kernel: [  799.548188] vi 54080000.vi: tegra_channel_error_status:error 20022 frame 84
May  4 06:54:37 tegra-ubuntu kernel: [  799.643864] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.643874] video4linux video0: frame start syncpt timeout!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.665670] vi 54080000.vi: tegra_channel_error_status:error 20022 frame 90
May  4 06:54:37 tegra-ubuntu kernel: [  799.775730] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.911816] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  800.043797] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  800.176227] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  800.307984] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:38 tegra-ubuntu kernel: [  800.443861] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:38 tegra-ubuntu kernel: [  800.575928] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:38 tegra-ubuntu kernel: [  800.708438] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:38 tegra-ubuntu kernel: [  800.844025] video4linux video0: MW_ACK_DONE syncpoint time out!0

ShaneCCC · May 4, 2018, 7:15am

Looks like Tegra doesn’t support RGB-IR sensors.

3dimi · May 4, 2018, 7:31am

Hi Shane,

The application is only streaming the raw data without debayering. Shouldn’t Tegra be able to support this?

ShaneCCC · May 4, 2018, 7:39am

Does the data format and data size the same with any of the bayer format in the TX1 support list in the TRM?

3dimi · May 7, 2018, 12:33am

Hi Shane,

Yes, 12-bit raw is supported in TX1.

ShaneCCC · May 7, 2018, 5:48am

OK, then you may need to probe the MIPI timing to make sure it match the MIPI spec.

3dimi · May 7, 2018, 6:30am

Hi Shane,

We will also check in the deserializer spec if there are specific settings that we missed.

Just to clarify, if it is a MIPI timing or any hardware timing issue, wouldn’t it also show up as a problem in the argus_camera or nvgstcapture stream? When we use those two, we can see continuous streaming with no blank frames. We only see an issue in V4L2. Also, in V4L2, we do get a few good frames which means the data has been parsed correctly.

ShaneCCC · May 7, 2018, 6:35am

Does the 28.2 have the same symptom? argsu/gstreamer have no problem but v4l2 utility.

3dimi · May 7, 2018, 7:50am

Hi Shane,

We only observed that in 28.1 where gstreamer and argus are both working properly and v4l2 has issues (only buffer.index = 0 is used).

In 28.2, gstreamer and argus are no longer working (no video stream) while v4l2 still has issues.

What do these observations mean?

Topic		Replies	Views
Vi2 CSI_A syncpt timeout Jetson TK1	37	4808	January 2, 2018
AR0144 on TX2: fence timeouts, MIPI CRC? Jetson TX2	15	3255	October 18, 2021
Tx2 CSI grab data from FPGA without I2c Jetson TX2	16	1243	October 18, 2021
Problems on porting ov9281 csi camera on our carrier board Jetson TX2 camera	51	3472	April 21, 2022
Tegra Multimedia APIs could get raw image via v4l2 only at 15fps Jetson TX2	77	9192	October 18, 2021
How to grab pre-configured csi video stream without I2C? Jetson TX2	67	11906	April 16, 2018
Multi_camera "No cameras available" Jetson Nano camera	58	4341	October 15, 2021
Same camera on same carrier board, TX1 works fine,but TX2 can't open camera Jetson TX2	17	1646	October 18, 2021
NVCSI - Image corruption with L4T r28.2 Jetson TX2	16	2838	October 18, 2021
Two CSI cameras each work individually, but not at the same time Jetson TX2	7	857	October 18, 2021

Camera outputs to 1 buffer only

Related topics