Jetson Xavier CSI2 non-image data capture (packet data type 0x12)


I am using Jetson Xavier AGX loaded with JetPack 32.1 and a custom image sensor connected to it over MIPI CSI2.
The sensor is configured to send non-image data (statistics) using CSI2 packets with data type 0x12.
I see that libargus seems to proide methods to grab such non-image data (CaptureMetadata object - .
But I cannot use the libargus for performances and control reasons.

Since libargus sits on top of the kernel drivers, I just need to understand the interface it uses with the kernel to grab these non-image data.

After several hours of code digging inside NVIDIA & v4l linux kernel drivers, I do not see evidence of how to grab the content of these 0x12 CSI2 data packets to the userland (using some ioctl or any other kernel communication means on a device).

Can anyone please help me and eventually give me orientations on how to interface the NVIDIA/v4l drivers to grab CSI2 0x12 non-image data ?

Thanks in advance,

Moving to Jetson AGX Xavier forum for resolution.

hello a.gilles,

may I know what’s your use-case.
you may start with v4l2 standard controls besides working with libargus,
for example,
you may refer to Applications Using V4L2 IOCTL Directly to verify basic functionality during sensor bring-up.

Hello JerryChang

Thank for fast feedback !

I design a testbench to validates the functions of a MIPI sensor.
Basic functionalities were covered already, like configuration and streaming in different video resolutions.
The only function that needs to be closed is the MIPI 0x12 non-image data packets retrieval.
I am sure that these 0x12 MIPI packets are sent by the sensor. They are surely received by the AGX MIPI interface as well.

The link you provided covers basic functionalities using IOCTL, I do not see the non-image data subject being covered.

Is the libargus source code available ?


hello a.gilles,

please check an example of a device tree node for the IMX185 V4L2 sensor driver,
there’s sensor active region as 1920x1080, and also with an extra embedded metadata rows for each frame.
it’s pixel parser to handle both (embedded data & image data) from single CSI camera.
for example,

imx185_a@1a {
    compatible = "nvidia,imx185"; 
    mode0 {
        active_w = "1920";
        active_h = "1080";
        embedded_metadata_height = "1";

please access Xavier TRM, you may check [Figure 7.6 External Connectivity Diagram] for reference.
please also refer to [Property-Value Pairs] in Sensor Software Driver Programming Guide for the description of embedded_metadata_height.

I think it might not works since you would like to handle non-image data packets, which means you should configure invalid active regions into device tree.

there’re some MMAPI samples, libargus source code are not available

Hello JerryChang,

I went through the embedded_metadata_height inside the DT already.
To my understading, embedded_metadata_height relates to non-image data embedded into each frame. at least this is what I understood from the drivers source code. It does not seem to relate to the 0x12 MIPI non-image data packets.

I refer to kernel source file vi5_fops.c, function vi5_setup_surface() where I see that the embedded_data_height property (loaded earlier from DT) is used to setup

Can you confirm that device-tree “embedded_metadata_height” has no link with the MIPI 0x12 CSI2 packets ?


hello a.gilles,

CSI data types of 0x12 shows the embedded data, it should not be a problem.
we don’t have use-case to process non-image data packets.
as I mentioned in post #6, it’s pixel parser to handle both (embedded data & image data) from single CSI camera.

may I know what’s your VI tracing logs reports.
you may gather those as following,
for example,

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace

there’s DTYPE to identify NVCSI datatypes.
please also access L4T sources,
you may also check header file, camrtc-capture.h to decode the NVCSI datatypes from VI tracing logs.
for example,