Extracting Embedded Data (EBD) from MIPI Frames on Jetson AGX Orin

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

Dear all and the Jetson Team,

We are currently working on extracting EBD (embedded data) from a MIPI frame on the Jetson AGX Orin using JetPack 6.0.

  • The embedded data height is 1 px, and the width is 1920 px.

  • We are able to receive MIPI frames without EBD successfully; however, once we enable the one-line EBD, the VI capture consistently times out.

We would greatly appreciate any insights or successful experiences related to EBD, as well as suggestions on how we might further investigate the root cause or required driver modifications.

Thank you very much for your assistance.

5

hello kaichu.wu,

may I know what’s your capture pipeline? please refer to Camera Architecture Stack,
for instance, are you going to extract embedded metadata via libargus, or, you’re using standard v4l2 IOCTL?
FYI, here’s Libargus Camera API, Argus::Ext::ISensorPrivateMetadata::getMetadata.

6

Dear Jerry,

Thank you for your quick response. We are targeting the v4l2 ioctl pipeline.
To provide a bit more context, we are streaming MIPI data from GMSL2 cameras with an ISP on the camera side, and the embedded data (EBD) is also generated by the camera-side ISP.

7

hello kaichu.wu,

embedded line is transferred with data type 0x12 in the packet header, it’s processed by pixel parser individually.
there’s chan->emb_buf; to allocates private buffer for embedded data storage. you may retrieved via user-space.
please see-also VI driver for more details.
for instance,
$public_sources/kernel_src/kernel/nvidia-oot/drivers/media/platform/tegra/camera/vi/vi5_fops.c

static void vi5_setup_surface(...)
{
        ...
        if (chan->embedded_data_height > 0) {
                desc->ch_cfg.embdata_enable = 1;
                desc->ch_cfg.frame.embed_x = chan->embedded_data_width * BPP_MEM;
                desc->ch_cfg.frame.embed_y = chan->embedded_data_height; 
                
                desc_memoryinfo->surface[VI_ATOMP_SURFACE_EMBEDDED].base_address
                        = chan->emb_buf;
8

Hello Jerry,

Thank you for the great support. We have successfully received the video data and EBD after applying the proper MIPI header configuration.

tmp
tmp457×225 42.5 KB