nvvidconv doesn’t handle 24 bits formats so far, AFAIK. So you would have to convert with nvvidconv into RGBA or BGRx and use videoconvert for removing the extra 4th byte.
For high resolutions/framerates, an alternative to opencv videoio is to use jetson-utils (if you mind GPU processing, you may check this post. For python it might be more complex).