The problem may be that your client pipeline uses decodebin, that would pick up nvv4l2decoder for H265 decoding on Jetson, but this decoder outputs into NVMM memory that cannot be handled by xvimagesink.
You may try adding nvvidconv between decodebin and xvimagesink.
my records indicate that the following pipeline [ credits to @Honey_Patouceul ] worked at some point of time
./test-launch "ximagesrc use-damage=0 ! nvvidconv ! omxh264enc ! video/x-h264, profile=baseline ! h264parse ! video/x-h264, stream-format=byte-stream ! rtph264pay name=pay0 pt=96 "
client side
gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp, media=video, encoding-name=H264 ! rtph264depay ! h264parse ! avdec_h264 ! xvimagesink
However, tests were conducted both server/client on the same AGX