How to stream Y12 (12-bit grayscale) camera on Jetson Orin NX with JetPack 5.1.4

Robotics & Edge Computing Jetson Systems Jetson Orin NX
, ,
1

Hi,

I am working with a monochrome camera on Jetson Orin NX running JetPack 5.1.4. The camera supports Y12 (12-bit grayscale) format, which I verified using v4l2-ctl:

v4l2-ctl -d /dev/video0 --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'Y12 ' (12-bit Greyscale)
		Size: Discrete 640x480   Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1280x720  Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1920x1080 Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 3840x2160 Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 5120x2880 Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 5120x3840 Interval: Discrete 0.017s (60.000 fps)

Currently, I am able to stream Y8 using the following GStreamer pipeline:

gst-launch-1.0 v4l2src device=/dev/video0 !
video/x-raw,format=GRAY8,width=640,height=480,framerate=60/1 !
nvvidconv !
video/x-raw,format=I420 !
fpsdisplaysink text-overlay=true video-sink=autovideosink sync=true

However, I would like to stream Y12 format from the same camera.

My questions:

  1. Does GStreamer on Jetson Orin NX (JetPack 5.1.4) support Y12 format directly?

  2. What is the correct GStreamer pipeline to stream Y12?

  3. Should Y12 be treated as GRAY16_LE in GStreamer?

  4. Does nvvidconv support Y12 / GRAY16_LE conversion?

  5. If GStreamer does not support Y12 directly, what alternative applications or approaches can be used to stream or visualize Y12 data?

Any guidance or example pipelines would be greatly appreciated.

Thanks!

4

Suppose NVCSI/VI don’t support Y12, How do you modify the VI driver to get it.

BTW, You can check what the element format by gst-inspect-1.0 command like.

gst-inspect-1.0 nvvidconv
5

Hi @ShaneCCC ,

Thanks for your suggestion.

I have already modified the VI driver to add Y12 format support. I applied the following changes:

1. Added Y8 and Y12 support in camera_common.c

{
    MEDIA_BUS_FMT_Y8_1X8,
    V4L2_COLORSPACE_RAW,
    V4L2_PIX_FMT_GREY,
},
{
    MEDIA_BUS_FMT_Y12_1X12,
    V4L2_COLORSPACE_RAW,
    V4L2_PIX_FMT_Y12,
},

2. Added format mapping in sensor_common.c

else if (strncmp(pixel_t, "bayer_gray8", size) == 0)
    *format = V4L2_PIX_FMT_GREY;
else if (strncmp(pixel_t, "bayer_gray12", size) == 0)
    *format = V4L2_PIX_FMT_Y12;

3. Added format in vi5_formats.h

TEGRA_VIDEO_FORMAT(RAW8, 8, Y8_1X8, 1, 1, T_R8, RAW8, GREY, "GRAY8"),
TEGRA_VIDEO_FORMAT(RAW12, 12, Y12_1X12, 2, 1, T_R16, RAW12, Y12, "GRAY12"),

After applying this patch and rebuilding the kernel, I am able to successfully enumerate and capture using Y12:

v4l2-ctl --list-formats-ext
[0]: 'Y12 ' (12-bit Greyscale)

So VI capture path is working correctly.

However, when checking nvvidconv using gst-inspect:

gst-inspect-1.0 nvvidconv

I can see support for:

GRAY8
I420_12LE
P010_10LE
...

but **Y12 is not listed as a supported input format
**
Since nvvidconv does not support Y12, what is the recommended way to stream Y12 using GStreamer on Jetson?

Please advise the recommended approach.

Thanks.

6

You may check if any 3rdParty software element for your request.

Thanks

7

Hi @ShaneCCC ,

Is there any default NVIDIA application or tool available in JetPack that can stream or display Y12 (12-bit grayscale) format?

Thanks.

9

Hi,
Orin hardware doesn’t support Y12. Please convert Y12 to supported format through software converter, such as videoconvert plugin in streamer.

10

Hi @DaneLLL ,

Thanks for the clarification.

Could you please provide an example GStreamer pipeline using the videoconvert plugin to convert and stream Y12 format on Jetson Orin NX?

This would help me verify and test the conversion on my setup.

Thanks.

11

Hi,
I think you’re right. Y12 is not defined in
https://gstreamer.freedesktop.org/documentation/video/video-format.html#GstVideoFormat

So cannot stream the format in gstreamer.

12

Hi @DaneLLL ,

Thanks for the confirmation.

Is there any possible way to stream Y12 format on Jetson Orin NX, either through format conversion (for example, converting to GRAY16_LE ) or using any NVIDIA-supported or third-party application?

Thanks.

13

Hi,
There is no existing sample and a possible solution is

  1. Capture frame data into CUDA buffer
  2. Implement format conversion through CUDA proramming

For capturing frame data into CUDA buffer, you can refer to the sample:

/usr/src/jetson_multimedia_api/samples/18_v4l2_camera_cuda_rgb

And implement the format conversion.