Hi,
Not sure why the API does not work. This would need other users to share experience.
There is a hardware converter in Jetson platforms and we have software implementation in gstreamer and jetson_multimedia_api. Would suggest use the engine for downscaling, cropping, format conversion. It is nvvidconv plugin in gstreamer, and NvBufferTransform() in jetson_multimedia_api.
There are samples for running with OpenCV. Please check
Nano not using GPU with gstreamer/python. Slow FPS, dropped frames - #8 by DaneLLL
LibArgus EGLStream to nvivafilter - #14 by DaneLLL