I don’t think that nvvidconv is the culprit… using :
... alpha plugin -> nvvidconv as RGBA into NVMM memory -> nvvidconv as RGBA into system memory -> compositor ...
works fine.
For your display issue with compositor, it may be related to pixel-aspect-ratio. Add verbose flag (-v) to gst-launch command so that you can get more insight about it.
You may force it to 1/1 in caps before compositor inputs (nvvidconv may also do the job such as here).
On compositor output, you may also resize to your screen resolution and add pixel-aspect-ratio=1/1 into caps.