X264enc plugin allows rtsp reconnection, but when uses nvh264enc rtsp can be played only once

This is my pipeline, after every primary RTSP disconnection from the client application like VLC, the next ones are not connecting.
GST_RTSP_MEDIA_STATUS_UNPREPARING status is read from the rtsp_media associated with it. do_create_element and do_configure are called even at the second connection but are not able to connect it. It is working perfectly fine if x264enc is used instead of nvenc.

This is a big blocker for us. Any idea how to proceed.

Hi Friends, any idea on this? it seems the port number is busy, starting from the second time