Black bars on video capture

Hi,
I was working on a driver porting from Jetpack 4.6 to 5.1.3, using an imx219 and serdes setup for the Xavier NX.
On Jetpack 4.6 capture looks fine, but when capturing on Jetpack 5.1.3, I get something like the attached video(1640x1232) for certain resolutions, for example 1280x720 and 640x480 look fine, but 1080 or 1640x1232 the problem appears.
The register tables are the same, the device trees properties(for capture) are the same, for the exception of serdes clk. Is there something that I’m missing. Some property that was changed or some new property that needs to be added? I played around with the pix_clk and line_length but only managed to make the capture zone smaller.
I’m testing with this:

gst-launch-1.0 nvarguscamerasrc sensor-id=7 ! "video/x-raw(memory:NVMM),width=1640,height=1232,framerate=30/1" ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=vid.mp4 -e

Also verified with different sensors and serdes ports.
Regards,
Andres

Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.c om/
Website: www.ridgerun.com

Could you confirm preview instead of video encoder.

Thanks

Hi,
Yes I get the same result using the following pipeline:

DISPLAY=:0 gst-launch-1.0 nvarguscamerasrc sensor-id=7 ! "video/x-raw(memory:NVMM),width=1640,height=1232,framerate=15/1" ! queue ! nvvidconv ! autovideosink

See the attached images, for 720 it looks fine, but for 1640 it doesn’t.

Regards,
Andres


Did you attached wrong picture? Can’t see the problem.

Yes those are the correct images.
sample_720 is how it should look like(using 720 capture resolution), the capture window and the capture feed are the same size, check where the window’s exit and minimize buttons are and the ubuntu app bar, for corner reference.
If you look at sample_1640, all of that black space is the videosink’s window, again look at the minimize button and the app bar for corner reference, but the capture is only a fraction of it, same as in the video. The capture should be full screen if it was really 1640, but its not, same as the video.

Where’s black bars?

I marked them in white and also marked the preview window outline in red so maybe its easier to see the issue. Let me know if you still can’t see the issue.

I would suspect it could be display sink problem. Maybe confirm by argus_camera or xvimagesink or nveglglessink.

But I see the same using the encoder->filesink, shouldn’t that discard the videosink as being the problem?
But I’ll try with the argus sample to triple check

@ShaneCCC, triedwith other displaysinks, and the same can be seen. Tested with the 09_argus_camera_jpeg/ sample and can’t get anything, getting a black screen. Something to note is that I cannot get capture with this:

v4l2-ctl --set-fmt-video=width=1648,height=1230,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap -d /dev/video7

And on the dmesg I see:

tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 8192

The later worked fine on the JP4.6 setup.

Could you check if the problem depend on the ubuntu OS?
The J4.x is ubuntu 18.04 and J5.x is 20.04

I’m not sure I understand your question. So to go back to the beginning:

  • Capture works fine with nvarguscamerasrc and v4l2-clt with Jetpack 4.6 for Xavier NX for all resolutions
  • I see the described issues with Jetpack 5.1.3 for both, nvarguscamerasrc and v4l2-ctl with the same Xavier NX
  • I do not see the issues on JP5.1.3 if the 720 or the 480 resolutions are used when using nvarguscamerasrc, v4l2-ctl fails for all resolutions.
  • Exact same hardware flashed with JP4.6 works fine.
  • I also have another set of hardware where I made the same tests and the results are the same(all fine with JP4.6, issues with JP5.1.3). So its not hardware related

The problem shows up only on JP5.1.3. The setup is with a FPDLink-iii bridge with imx219 cameras. The serdes_clk was tuned to match the new spec, and multiple capture can be done, but I see the same black bars on all the streams. If I move the serdes_clk down, multiple simultaneous capture cannot be done, and if I move the serdes_clk up, there is no capture at all. As I said at the start I’m not sure if I’m overlooking a device tree property that was added or changed between those releases.

I would like the check the black bar problem if relative with the OS.
Do you check the video file from JP4 and JP5 by host machine like windows system?

For the v4l2-ctl problem it could be the serdes_pix_clk_hz problem. We don’t see the problem with PI imx219 without FPDLink.

Yes I played the video file with gst-play-1.0, ubuntu video player and vlc. On all three I see the same issue.
For a serdes setup on JP5.1.3 the main changes for serdes are serdes_clk and deskew signal, those were added on the driver/dtb. Is there any other property that was changed/added.

I don’t think so.

Could you play on windows system for the video from different JP?

I can confirm the same hapens in Windows player(added white square to cover sensitive data):


The JP4 video, looks fine and fills up the whole window:

Hi,
Please try the samples and see if saved JPEGs and h264 stream have the issue:

/usr/src/jetson_multimedia_api/samples/09_argus_camera_jpeg/
/usr/src/jetson_multimedia_api/samples/10_argus_camera_recording/

If it is present in both gstreamer and jetson_multimedia_api, it is more like the frame data from the camera source has the black bars already.

Solved. The issue was due to a mismatch on the mode declaration on the drivers and the modes on the device tree. Now all modes work and look fine.
PS: This tool is easier to use and doesn’t require the multimedia extra install as the argus samples. At least on jp511 and jp513 its available by default.