Roi encoding

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

I have AGX Orin and use NvVideoEncoder API to do HEVC encoding.

I like to enable ROI encoding. I have the following questions on ROI encoding:

  1. How many ROIs can be defined?

  2. How to enable it and how to specify ROI parameters? Is there a C++ sample code of ROI encoding available?

Thanks

2

*** Please note that this reply is generated by LLM automatically ***

I’d be happy to help you with your questions about ROI (Region of Interest) encoding on the Jetson AGX Orin.

1. How many ROIs can be defined?

The Jetson AGX Orin supports up to 16 virtual channels with ISP (Image Signal Processor) or 24 virtual channels without ISP. Each virtual channel can be configured to have its own ROI settings. However, the actual number of ROIs that can be defined depends on the specific use case and the available resources on the device.

In general, the Jetson AGX Orin can support multiple ROIs per virtual channel, but the exact number may vary depending on the resolution, frame rate, and other factors. For example, if you’re using a high-resolution camera with a high frame rate, you may be able to define fewer ROIs compared to a lower-resolution camera with a lower frame rate.

2. How to enable ROI encoding?

To enable ROI encoding on the Jetson AGX Orin, you’ll need to use the NVIDIA Video Encoder (NVE) API, which provides a set of APIs for encoding video streams. Specifically, you’ll need to use the NvVideoEncoder API to create an encoder instance and configure it to use ROI encoding.

Here’s a high-level overview of the steps involved:

  1. Create an NvVideoEncoder instance using the NvVideoEncoder::create() function.
  2. Configure the encoder instance to use ROI encoding by setting the roi_enabled property to true.
  3. Define the ROI parameters using the NvVideoEncoder::set_roi() function.
  4. Start the encoding process using the NvVideoEncoder::encode() function.

3. How to specify ROI parameters?

To specify ROI parameters, you’ll need to use the NvVideoEncoder::set_roi() function, which takes a NvRoi struct as an argument. The NvRoi struct contains the following fields:

  • x: The x-coordinate of the top-left corner of the ROI.
  • y: The y-coordinate of the top-left corner of the ROI.
  • width: The width of the ROI.
  • height: The height of the ROI.

You can specify multiple ROIs by creating an array of NvRoi structs and passing it to the NvVideoEncoder::set_roi() function.

4. Is there a C sample code of ROI encoding available?

Yes, NVIDIA provides a sample code for ROI encoding on the Jetson AGX Orin, which is part of the NVIDIA Video Encoder (NVE) API. You can find the sample code in the samples directory of the NVE API package.

Here’s an example code snippet that demonstrates how to enable ROI encoding using the NVE API:

#include <nvvideoencoder.h>

int main() {
  // Create an NvVideoEncoder instance
  NvVideoEncoder *encoder = NvVideoEncoder::create();

  // Configure the encoder to use ROI encoding
  encoder->set_roi_enabled(true);

  // Define the ROI parameters
  NvRoi roi;
  roi.x = 100;
  roi.y = 100;
  roi.width = 200;
  roi.height = 200;

  // Set the ROI parameters
  encoder->set_roi(&roi);

  // Start the encoding process
  encoder->encode();

  return 0;
}

Note that this is just a simplified example code snippet, and you may need to modify it to suit your specific use case.

I hope this helps! Let me know if you have any further questions.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

4
  1. I don’t use Orin ISP. My encoder is one channel at 5760x1080, fps-30.
    How many ROIS can be used in my use case?
    2)I saw the defintion of typedef struct v4l2_enc_ROI_param_

{

 struct v4l2_rect  [ROIRect](https://docs.nvidia.com/jetson/archives/r35.5.0/ApiReference/structv4l2__enc__ROI__param__.html#a98413df4e7e00ebf2f0bd011d9ca9283);

 \__s32   [QPdelta](https://docs.nvidia.com/jetson/archives/r35.5.0/ApiReference/structv4l2__enc__ROI__param__.html#a46819a7b8c6a4573cc985a752f0877a0);

} v4l2_enc_ROI_param;

Can you explain the meaning of QPDelta and what are ranges of this parameter?

6

Him
Please try roi.cfg the user shared in
How to use ROI encoding on XavierNX

And please refer to the explanation:
How to use setROIParams function on Jetson to set dynamic RoI encoding? - #11 by DaneLLL

7

I make changes to enable ROI and have an encoded HEVC video.

Is there a tool available which I can verify ROI is working properly by examining QP values of the encoded video.?

Thanks,

8

Hi,
It seems there is no free tool to check QP value of each macroblock. A user has mentioned a tool in
Jetson multimedia API sample 01 video_encode - #12 by DaneLLL

Or see if other users can suggest any other tool.

9

It is difficult to see whether ROI is working or not with CBR mode

Do you know how to configure encoder with constant QP so I can see whether QP value in the ROI region is different from the rest of frame easily?

10

DaneLLL.

My frame resolution is 5760x1080 and I define 8 ROI regions, but only the first one is used in populate_roi_Param:

VEnc_ROI_params->num_ROI_regions=1;

"ROI": [
    {
        "dq": "5",
        "x": "0",
        "y": "810",
        "width": "5760",
        "height": "270"
    },
    {
        "dq": "0",
        "x": "0",
        "y": "0",
        "width": "0",
        "height": "0"
    },
    {
        "dq": "0",
        "x": "0",
        "y": "0",
        "width": "0",
        "height": "0"
    },
    {
        "dq": "0",
        "x": "0",
        "y": "0",
        "width": "0",
        "height": "0"
    },
    {
        "dq": "0",
        "x": "0",
        "y": "0",
        "width": "0",
        "height": "0"
    },
    {
        "dq": "0",
        "x": "0",
        "y": "0",
        "width": "0",
        "height": "0"
    },
    {
        "dq": "0",
        "x": "0",
        "y": "0",
        "width": "0",
        "height": "0"
    },
    {
        "dq": "0",
        "x": "0",
        "y": "0",
        "width": "0",
        "height": "0"
    }
]

When I use elecard stream eye app tocheck qp values for each CTUs inm a frame, qp values are the same for all CTUs within each frame. It is quite clear that ROI encoding is not working.
How to debug this?

11

Hi,
Please try the command with the roi.cfg:
How to use ROI encoding on XavierNX

This is validated. Please make sure you can run this and get correct result first.

And would suggest use latest Jetpack 6.2.1 r36.4.4

12

My Orin jetpack version:
Package: nvidia-jetpack
Version: 5.1.2-b104
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194

Will this version work?

13

Danelll

I am able to see ROI encoding working with this exmaple on my Orin:

However, my own encoder app does not work. I review flows between mine and reference one, they are almost identical except I have slice encoding features enabled.
I may overlook some minor details between these 2. Do you know anything settings I need to pay attention with ROI encoding?

14

Hi,
Please disable slice level encoding in your application and check if ROI encoding works.

15

Is quite difficult to disable slice_encoding for me.

Can you show me how to enable slice encoding on 01_video)encode to verify roi encoding still works not?

Also I found v4l2_buf.index value didn’t increase in my app as the one in 01_video_encode. why is that?

16

danelll,

ROI on 01_video_encode  still works OK with slice encoding enabled
17

Hi,
You may refer to 01 sample to debug your application.

18

I will review 01 sample code craefully.

I have another question of encoder HW presets.

There are 4 Presets on AGX Orin, as below:

V4L2_ENC_HW_PRESET_ULTRAFAST;

V4L2_ENC_HW_PRESET_FAST;

V4L2_ENC_HW_PRESET_MEDIUM;

V4L2_ENC_HW_PRESET_SLOW;

Where can I find definitions of 4 presets?

I want to create my own preset derived from one of these 4 presets.

Thanks,

19

Danelll

 The reason I am asking for these 4 presets is because I found V4L2_ENC_HW_PRESET_FAST is not realtime for 5760x1080. 
I want to refine V4L2_ENC_HW_PRESET_FAST settings to make it run realtime

FC
20

Hi,
The presets are hard defined and cannot be customized. If setting to V4L2_ENC_HW_PRESET_FAST cannot meet the target performance, please set to V4L2_ENC_HW_PRESET_ULTRAFAST.

21

Yes, I am currently using V4L2_ENC_HW_PRESET_ULTRAFAST, but the quality is not good enough.

I am hoping V4L2_ENC_HW_PRESET_FAST can improve video quality.. Unfortunely, it is not realtime for 5760x1080 at 30fps.

What I want to know the difference of the following items between ULTRAFAST and FAST so I can set them myself:

1)motion search range

2)number of reference number

3)motion search algorithm

Is there a way to dump values of the above 3 items in my C/C++ application?

Thanks

22

Hi,
Performance and image quality are tradeoff. You would need to use V4L2_ENC_HW_PRESET_ULTRAFAST. One more thing you may try is to configure CBR + virtual buffer size. Please refer to the example:
Random blockiness in the picture RTSP server-client -Jetson TX2 - #5 by DaneLLL