Try this pipeline, It works for me. I used videotestsrc to simulate a 60fps camera and queue to simulate nvinfer.
Finally, I want to confirm something: does the inference rate of nvinfer (60fps vs. 30fps) significantly affect the output? You might consider adding videorate downstream of v4l2src.
gst-launch-1.0 -v \
videotestsrc is-live=true \
! video/x-raw,format=I420,width=1280,height=720,framerate=60/1 \
! nvvideoconvert \
! 'video/x-raw(memory:NVMM),format=NV12' \
! queue \
! videorate \
! 'video/x-raw(memory:NVMM),framerate=30/1' \
! tee name=t \
t. ! queue ! fpsdisplaysink video-sink=fakesink text-overlay=false sync=false \
t. ! queue ! nvv4l2h264enc bitrate=4000000 insert-sps-pps=true idrinterval=30 \
! h264parse config-interval=-1 \
! rtph264pay pt=96 config-interval=1 name=pay0 \
! udpsink host=127.0.0.1 port=5000 sync=false async=false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 67, dropped: 0, current: 29.99, average: 32.75
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 83, dropped: 0, current: 30.00, average: 32.18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 99, dropped: 0, current: 30.01, average: 31.81
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 115, dropped: 0, current: 30.00, average: 31.55
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 130, dropped: 0, current: 30.00, average: 31.36
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 145, dropped: 0, current: 29.98, average: 31.21
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 160, dropped: 0, current: 29.99, average: 31.09
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 176, dropped: 0, current: 30.02, average: 30.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 191, dropped: 0, current: 29.99, average: 30.91
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 207, dropped: 0, current: 30.02, average: 30.84
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 222, dropped: 0, current: 30.00, average: 30.78