NvBufferTransform Failed

Hi,

I met this error in our gstreamer-base APP.
The issue was hit when we try to run a h264decode + render pipeline while there were already many pipelines running (many nvvidconv and nvv4l2h264enc, nvv4l2decoder plugins used).
It seems somehow the existing pipelines were suddenly impacted by the new pipeline.

So I would like to ask what these errors mean?
Could it be possible that we hit performance limit?
Thanks.

SYNC_IOC_FENCE_INFO ioctl failed with 9
2022-08-23 14:06:41.553338 0x7f2a4388a0 nvvidconv ERROR: gst_nvvconv_transform: NvBufferTransform Failed
SYNC_IOC_FENCE_INFO ioctl failed with 9
2022-08-23 14:06:41.553905 0x7e180615e0 nvvidconv ERROR: gst_nvvconv_transform: NvBufferTransform Failed
2022-08-23 14:06:41.554324 0x7e180615e0 basesrc WARN : error: Internal data stream error.
2022-08-23 14:06:41.554366 0x7e180615e0 basesrc WARN : error: streaming stopped, reason error (-5)
2022-08-23 14:06:41.555828 0x7e18061720 EMS-BusMonitor WARN : WP-Z510_Video_0400-0000 Error received from element WP-Z510_Video_0400-0000_nvv4l2camerasrc_0: Internal data stream error.
2022-08-23 14:06:41.555918 0x7e18061720 EMS-BusMonitor WARN : WP-Z510_Video_0400-0000 Debugging information: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline884/GstNvV4l2CameraSrc :WP-Z510_Video_0400-0000_nvv4l2camerasrc_0:
streaming stopped, reason error (-5)
SYNC_IOC_FENCE_INFO ioctl failed with 9
NvMMLiteVideoEncDoWork:NvMSEncConvertSurfaceFormat failed
SYNC_IOC_FENCE_INFO ioctl failed with 9
2022-08-23 14:06:41.557014 0x7e84033e80 nvvidconv ERROR: gst_nvvconv_transform: NvBufferTransform Failed
SYNC_IOC_FENCE_INFO ioctl failed with 9
2022-08-23 14:06:41.558602 0x7ef0175940 nvvidconv ERROR: gst_nvvconv_transform: NvBufferTransform Failed
NvVideoEncTransferOutputBufferToBlock: DoWork failed line# 674
NvVideoEnc: NvVideoEncTransferOutputBufferToBlock TransferBufferToBlock failed Line=685

Hi,
Please run VIC engine at maximum clock and check if it helps:
Nvvideoconvert issue, nvvideoconvert in DS4 is better than Ds5? - #3 by DaneLLL

It is the engine of executing NvBufferTransform(). For heavy loading it is better to keep it at maximum clock.

Unfortunately, we already applied this trick by default.
But we still hit this issue.

Hi,
Another possibility is memory leak happens and triggers the issue. Do you decode UDP sources or RTSP sources in your use-case? Or what are the input sources?

One thing I noticed is that VIC frequency hit 99% all the time.
There are 9 nvv4l2h624enc (1080P,20fps) + 8 nvv4l2decoder + 50 nvvidconv (most of them just pass through) plugins used.
Will nvv4l2h264enc use VIC engine?
Is it possible that my issue is due to VIC engine limit?