How to implement low latency video display pipleline on PC?

I am using gstreamer pipeline to form some a ultra low latency video streaming pipeline.

I know there are nvvideosink and nvoverlaysink on Jetson which can bypass x.org to display video streaming content.

Is it possible to do it on PC with nvidia GPU? I normaly do it with glimagesink. I know there is some plugin like kmssink which bypass x.org, but it seems not working with nvidia propertiery driver.

And when I try to do hardware decoding with pipeline like :
nvdec ! glimagesink

The overall latency seems to be a little longer than software driven decoding pipeline :
avdec_h265 ! glimagesink

Is it due to decoded content go back force between GPU and CPU ?