ROI encoding with 01_video_encode sample

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
#1

Hello, I am not sure if this is the best forum for this question, but I would like to ask for tips and suggestions on using the ROI feature in the 01_video_encode sample application on the Jetson AGX Xavier.

I am encoding a 1920x1080 video as follows:

./video_encode traffic.yuv 1920 1080 H264 out.h264 --input-metadata --eroi -roi roi.txt -br 1000000

And the format of the roi.txt file is:

1 -5 0 0 256 256
…repeated for N frames…

From what I understand, a negative Qpdelta should improve the quality inside the ROI. I can see that the ROI setting is having some effect, but at -5 it is a subtle improvement. Reducing it further does show a nice improvement in the stationary parts of the video within the ROI, but in areas with motion there are a lot of bothersome artifacts.

I don’t have a lot of experience with all these encoder settings, so do I need to adjust some other properties in order to get good results with the ROI?

Thanks!

#3

Hi,
We are checking the functionalityof setting ROI. Will update.

There is a similar topic:
How to use ROI encoding on XavierNX

#4

Thanks for looking into this. We have been experimenting with this a bit more and found that if we lower the I-frame interval and the IDR interval, then we can reduce the artifacts inside the ROI. For example:

./video_encode traffic.yuv 1920 1080 264 out.h264 -br 1000000 -p baseline -rc cbr -ifi 4 -idri 30 --input-metadata --eroi -roi roi.txt

The problem, though, is that this increases the overall bitrate, so it seems to defeat the purpose of doing ROI encoding.

#5

Hi,
Could you try JP4.5? We have some fix in the release. See if it helps your usecase.

#6

Thanks, but I am already using JP4.5.

#7

Hi @DaneLLL

I am running into the same issue. I am using Jetpack 4.5. I built the same 01_encode_video sample. Then, I used the following command to encode a 1080p30 (420) video:
./encode_video <file_name> 1920 1080 H264 <file_name>.h264 -br 1000000 -p baseline -rc cbr -ifi 30 --cd --report-metadata --input-metadata --eroi -roi roi.cfg.

The roi.cfg contents are:
1 -15 1600 540 128 128

What I see is that the ROI region is not being updated every frame. If I set ifi to 30, then the ROI updates once per second (fps is 30). And the only way, as @phelps2020devel mentioned, to get the ROI to update is to decrease the i-frame interval. I have tried with other encoders and even at 500kbps, the ROI seems to update just fine every frame with similar settings. It is as though the nvVideoEncoder makes no use of P-frames for the ROI region …

Please advise.

#8

Hi,
Please try VBR mode. In CBR mode, obeying to bitrate is with highest priority. Other settings may be overwritten.

#9

Hi @DaneLLL

I had tried VBR and tried it again. The same issue persists. Can you please try it on your end?

Thanks.

#10

Hi,
Could you share how you verify the function? With –eroi -roi roi.cfg, I can see frame size increase for each frame. Since I frames are with larger size than P frames, probably the size increase is not significant in P frames.

#11

Hi @DaneLLL

Please see attached a couple of videos. You can see the ROI starting at col 1600, row 540 and size of 128x128. The rest of the frame is updating fine but the ROI is very jittery as though it only updates during an i-frame.
The videos were encoded using 1mbps and 4mbps (see names).
Once again, we have tried it with other encoders and this is not an issue i.e. the other encoder preserves details in the ROI (at the expense of the rest of the frame) but without compromising the update rate in the ROI itself…

Please let me know if you have anymore questions. This is of significant concern to us and we are somewhat relying on this to use the AGX for our product. Appreciate the help!

Thanks!

#12

Hi,
Looks like the bitrate setting is too small so it cannot encode in larger q value. Please try to set QP range:

$ ./video_encode a.yuv 1920 1080 H264 a.h264 -br 4000000 -p baseline -rc cbr -ifi 30 --cd --report-metadata --input-metadata --eroi -roi roi.cfg -MinQpP 1 -MaxQpP 40 -MinQpI 1 -MaxQpI 40

In this case, it may not be able to obey 4Mbps setting.

If you have tool to check the qp value of macroblocks and you observe the macroblocks not be encoded with lower q value(-15 in the roi.cfg), please share the method so that we can check. We have other users using JP4.5 and they confirm the function is good. If you still hit the issue, would need to investigate where the deviation is from.