Is there any way other than usb capture to get video stream from camera(hdmi/sdi) for use with CV libraries?

Hi,
It is possible. The capture goes through v4l2 interface. There are existing camera modules from our partners. Please check

For having optimal performance, using OpenCV may not be a good choice since there is limitation in Jetson hardware. You can take a look at
[Gstreamer] nvvidconv, BGR as INPUT - #2 by DaneLLL

So it is recommended use gstreamer or jetson_multimedia_api.
Welcome — Jetson Linux<br/>Developer Guide 34.1 documentation

Jetson Linux API Reference: Main Page | NVIDIA Docs