NVDEC hardware accelerated video decoding with fewer capture buffers?

I am using the hardware accelerated video decoding with NVDEC based on GStreamer. This is a wrapper around a V4L2 element. However, it asks the driver for the minimum number of capture planes (buffers) it should use, which returns 10. This request is an ioctl call with V4L2_CID_MIN_BUFFERS_FOR_CAPTURE.
The Jetson Multimedia example 00 for video decoding does the same.

I have many high-resolution streams with a low framerate (<10fps) to work with. What happens if I ignore this lower bound and give it fewer buffers? Will the hardware decoding break or corrupt memory?
Lowering it below 10 looks feasible to do with the Jetson Multimedia example 00.

Trying it experimentally, lowering the minimum to 6 (+1 extra capture buffer, so actually 7) buffers still works. Below that, it hangs immediately.
What is the risk of lowering the number of buffers like this?
And is there a way to achieve this change in the GStreamer nvv4l2decoder?

Low-level software stack and driver would need minimum buffer number for stable decoding tasks. If you allocate less buffers, it may not work properly in certain cases. We don’t suggest do this.