Access gstreamer output on GPU for fast inference

I am working with Gstreamer and Python to decode a video using HW acceleration on Nvidia GPU:

rtspsrc ! rtph264depay ! h264parse ! nvh264dec ! videoconvert ! appsink

The following pipeline can be access using OpenCV which will return a numpy array. However, since I need to process the resulting frame on the GPU, I was wondering if there is any component that I can use to replace appsink that would allow me to access the frame directly on the GPU using any cuda library (e.g. cupy, pycuda).