Nvvidconv/nvv4l2h264enc scaling artefacts

Robotics & Edge Computing Jetson Systems Jetson Nano
1

Hello,
I am using gst to transcode a video into multiple scales/bitrates. All is working great on the original size output (1080p), but on lower scale video, like 720p, elements with straight lines become warped, as if scaler would simply drop every n-th row from video instead of calculating averages.
So, the question is: Is it possible to change scaler behaviour, if yes - how?

These are example frames of the gst output. Notice the text and especially the logo on the right upper corner:

3

You may share your transcoding pipeline so that it is easier to reproduce.
Are you using nvvidconv for rescaling ? If yes, which interpolation-method values did you try ?

gst-inspect-1.0 nvvidconv
...
  interpolation-method: Set interpolation methods
                        flags: readable, writable, controllable
                        Enum "GstInterpolationMethod" Default: 0, "Nearest"
                           (0): Nearest          - Nearest
                           (1): Bilinear         - Bilinear
                           (2): 5-Tap            - 5-Tap
                           (3): 10-Tap           - 10-Tap
                           (4): Smart            - Smart
                           (5): Nicest           - Nicest
...
4

This is the pipeline that is being used:
gst-launch-1.0 souphttpsrc location=http://192.168.0.1/mpegts ! tsdemux name=input input.audio_0_013f ! parsebin ! queue ! tee name=a1 input.audio_0_01a3 ! parsebin ! queue ! tee name=a2 a1. ! queue ! mux. a2. ! queue ! mux. input.video_0_00db ! decodebin ! deinterlace ! tee name=video video. ! nvvidconv ! video/x-raw(memory:NVMM),width=1048,height=576,format=I420 ! nvv4l2h264enc maxperf-enable=1 control-rate=0 peak-bitrate=1635779 bitrate=1363149 preset-level=1 profile=High insert-sps-pps=1 ! h264parse ! queue ! mux. video. ! nvvidconv ! video/x-raw(memory:NVMM),width=654,height=360,format=I420 ! nvv4l2h264enc maxperf-enable=1 control-rate=0 peak-bitrate=1258291 bitrate=1048576 preset-level=1 profile=High insert-sps-pps=1 ! h264parse ! queue ! mux. mpegtsmux name=mux ! tcpserversink port=12345
Correct me if I’m wrong but, as I understand, this pipeline is using nvv4l2h264enc to resize the stream, not the nvvidconv. On nvvidconv I did not see scaling options, only cropping, so did not use interpolation-method.

5

I think that the following do rescaling with nvvidconv:

video. ! nvvidconv ! video/x-raw(memory:NVMM),width=1048,height=576,format=I420 ! nvv4l2h264enc
video. ! nvvidconv ! video/x-raw(memory:NVMM),width=654,height=360,format=I420 ! nvv4l2h264enc

so just add another interpolation method, such as Nicest for lowest resolution if this doesn’t hurt performance:

gst-launch-1.0 souphttpsrc location=http://192.168.0.1/mpegts ! tsdemux name=input input.audio_0_013f ! parsebin ! queue ! tee name=a1 input.audio_0_01a3 ! parsebin ! queue ! tee name=a2 a1. ! queue ! mux. a2. ! queue ! mux. input.video_0_00db ! decodebin ! deinterlace ! tee name=video video. ! nvvidconv ! video/x-raw(memory:NVMM),width=1048,height=576,format=I420 ! nvv4l2h264enc maxperf-enable=1 control-rate=0 peak-bitrate=1635779 bitrate=1363149 preset-level=1 profile=High insert-sps-pps=1 ! h264parse ! queue ! mux. video. ! nvvidconv interpolation-method=5 ! video/x-raw(memory:NVMM),width=654,height=360,format=I420 ! nvv4l2h264enc maxperf-enable=1 control-rate=0 peak-bitrate=1258291 bitrate=1048576 preset-level=1 profile=High insert-sps-pps=1 ! h264parse ! queue ! mux. mpegtsmux name=mux ! tcpserversink port=12345
6

Yes, this does solve the issue. Trying different interpolation-method levels does not seem to affect performance.
Can You describe major differences between these methods? Nearest and bilinear are quite self-explanatory, but other methods are a bit unclear, especially “nicest” does it use one of the above (0-4) method based on frame contents or is it just called “nicest” because some programmer was realy proud of his code?

7

Sorry, I fail to find a relevant doc link for these options.
Someone from NVIDIA may provide better information about these.

8

Hi,
Interpolation mode 5 is nicest. We may not be able to give further information about the algorithm. Please use it if it is good in your usecase. Thanks.

9

Alright, keep your secrets then. I’m just glad it works.

Thanks

10

Hi @Honey_Patouceul
Many thanks for your instant suggestion. Much appreciate your help.

11

Thanks @DaneLLL. Happy I’ve been helpful this time.

@jezaus.tevas,
I missed that before, you may add queue after outputs of video tee:

video. ! queue ! nvvidconv ! video/x-raw(memory:NVMM),width=1048,height=576,format=I420 ! nvv4l2h264enc
video. ! queue ! nvvidconv ! video/x-raw(memory:NVMM),width=654,height=360,format=I420 ! nvv4l2h264enc

this might save some synchro problems.

12

@Honey_Patouceul won’t this extra queueing reduce performance? I am already experiencing some odd behaviour with current pipeline, as in after extended period of uninterupted transcoding cpu and system memory usage increases constantly. At the beggining process uses ~20% cpu and ~10% ram, after ~6 hours cpu is increased to ~50% and ram to almost 40%.

13

Are you sure the increasing CPU and RAM usage is related to queue plugins ?

I just use queues for each parallel output of tee/demux and for each parallel input of mux as a general rule.
Without these, I think it may work in some cases, but may have synchronization issues.
Not sure how it can affect performance, but I think it may also allow a subpipeline to be run on a different core.

That’s only my own understanding, I may be wrong, you may have deeper knowledge than me about queue, but for your pipeline I’d try to remove the queues before tees a1 and a2 and add queues after video tee.

Someone with better knowledge may further comment. You may create a separate topic since this one is solved and will therefore get less visibility.