Hi,
Enabling the properties shall have latency enhancement:
// nvv4l2h264enc (poc-type=2)
maxperf-enable : Enable or Disable Max Performance mode
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
poc-type : Set Picture Order Count type value
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 2 Default: 0
// nvv4l2decoder
disable-dpb : Set to disable DPB buffer for low latency
flags: readable, writable
Boolean. Default: false
enable-max-performance: Set to enable max performance
flags: readable, writable
Boolean. Default: false
Please enable these and check.