Hi, I’m trying to use the setROIParams function on Jetson Xavier NX platform (should also work on other Jetson devices) to implement dynamic RoI encoding to encode a video. Our goal is to use Jetson’s hardware acceleration for video encoding and set different QP value for different regions. We used the function as in the API docs in our C++ code (ROIRect and QPdelta and maximum 8 RoI regions). But the problem is that the RoI encoding sometimes works and sometime fails (with no RoI encoded at all). We’ve tried different RoI location, area and QP value but still don’t understand when it work and when it fails. Some of our observations are:
The RoI region can’t be the same as frame size. It must be a little smaller than the frame size. But we don’t know how that exact value is defined.
The default QP value for unset region is unknown. Sometimes it seems their QP is very low (good quality) while sometimes the figure is almost static (unchanged).
Seems a single RoI region can’t be too large. Maybe it can’t cross multiple slices?
Sometimes the encoded video size is very large (3M+) and sometimes only 500KB with only a slight change of ROI region…
The official docs provide nearly no explanation for the usage of this function and there’s no way to debug since the values are directly set to the registers. All we know is the parameter format from notes of source code… Is there any one have used this function or give us some reference of how to use that? Thanks a lot!!!
If the frame does not need ROI, please not to set the flag. Please try this. For further issues, please provide a command of running 01_video_encode so that we can check.
Thanks for your reply. ROI encoding for unset frames remaining the same as previous one makes sense to me. What about QP for unset regions? E.g., for a image of 720*406, if I set the ROI as 1 51 0 0 100 100, all regions are blurred (unset region QP is 51), but if I set the ROI as 1 0 0 0 100 100, all regions are clear (unset region QP is 0). So the QP for unset regions are automatically calculated or we can set a default value for them?
Another thing I found weird is, for an image of size width 720 and height 406, if I set roiParams.txt to 2 0 0 0 360 406 51 360 0 360 406 (two parts, left QP is 0, right QP is 51), then the roi param has no effects, i.e., the result is a clear video (same as no roi params set). But if I change 406 (full height) to 380 (leave some space on vertical direction), and set roi params to 2 0 0 0 360 380 51 360 0 360 380, then the result is as expected, i.e., left part is clear (QP 0), right is blurred (QP 51) and a bottom slice is clear (the remaining 26 pixels height).
Does that mean we can’t set ROI region to overlap border pixels? If yes, why is the restriction and how many border pixels shall we leave out? If no, what’s the problem with our results? I appreciate it if you can have a try with these params to see if the problem is with our code or our usage of the API. Thanks a lot!
Hi,
The parameters are to set different QP values to the regions. Each frame is encoded with a qp value according to rate control mode(CBR or VBR) and bitrate:
POC must = frame# or field# for SNRs to be correct
--------------------------------------------------------------------------
Frame POC Pic# QP SnrY SnrU SnrV Y:U:V Time(ms)
--------------------------------------------------------------------------
00000(IDR) 0 0 16 4:2:0 256
00003( P ) 6 1 21 4:2:0 210
00001( b ) 2 2 18 4:2:0 262
00002( b ) 4 2 20 4:2:0 247
00006( P ) 12 2 21 4:2:0 212
00004( b ) 8 3 24 4:2:0 182
00005( b ) 10 3 24 4:2:0 186
For example, if you set a region in first IDR frame with QPdelta=-3, the maroblocks in the region are quantized with qp value=13.
Since we don’t test the case that regions are overlapped, please set the regions independently.
Thanks for your reply. I tried more parameter combinations and found the patterns:
The ROI region is actually working (but it can’t be too large). It works no matter it lie on the border or not.
The QP for unset regions are undetermined. Seems it’s related to the ROI region and QP we set.
Now the only question from my side is what is the largest size can an ROI be (or the sum of all ROI regions)? For an image of size 720*406, the result of set ROI params to 1 51 0 0 720 406 is the same as setting no ROI params, which means the setting is not working. But if I set the params to 1 51 0 0 720 380 (slightly shorter height), it’s perfectly working. So what is the largest size can ROIs be?
Hi，
Thanks for your answer, but I still a little confused. As YanyeKong said， “ for an image of size width 720 and height 406, if I set roiParams.txt to 2 0 0 0 360 406 51 360 0 360 406 (two parts, left QP is 0, right QP is 51), then the roi param has no effects”. The question is when we set QPdelta=0 for rect area “(0 0 360 406) (left top width height)” and QPdelta=51 for (360 0 360 406), the encoder output video qp value is not the value we set. That’s why anyeKong said the roi params has no effects. we wonder if there are some limits for roi parameters? for exsample, For one frame, all roi rects total width should be less than frame width?
Hi,
For setting ROI the internal rate control logic decides the QP value of each frame and you can set QPdelta to ROI blocks. It is not expected if you set ROI sa the whole frame. If your use-case is to control frame bits of whole frame, please configure v4l2_enc_frame_ext_rate_ctrl_params.
ROI works like:
QP value of the frame is decided by rate control logic per setting of rate control mode and bitrate
ROIs are compressed with the QP value + QPdelta
The other macroblocks are quantized with the QP value
We would expect ROI is smaller than the resolution. It may not work properly if it has same width/height as the resolution.