Hi,
Please refer to the topics:
Image format conversion with NvBufferTransform - #7 by DaneLLL
Image format conversion with NvBufferTransform - #7 by DaneLLL
Multimedia API capture with multi-planer color space - #3 by DaneLLL
You may check if the camera source can generate frame data fitting alignment of NvBuffer, and then customize nvv4l2camerasrc or 12_camera_v4l2_cuda to give it a try.
640x512 is not a larger resolution, so it should be fine to capture into CPU buffer first and then copy to NVMM buffer(NvBuffer). However, if you would like to do customization, please check the above topics.