Capture Video from a AV.io DVI-USB3 Device not working on TX2

A product I built for windows uses this device or their AV.io SDI-USB3 device to convert video, 1080p60, from DVI or SDI to USB3. This actually works very well, not expensive and fully controllable using the UVC driver in Windows.

Naturally, I would like this to work with my TX2 for a different project. Now that I removed the green lens protector from the included CSI camera that seems to be working fine. However, using VLC, I can connect to the device and it brings up a single image rather than video. I tried it with the imagenet-camera application in jetson-inference and it brings up an image rather than video. The image looks like the original frame has been appended to the right a few times.

I tried the camera and converter using VLC on my Ubuntu host machine (the NUC I flashed the TX2 from) and it works great! 1080p60 no problem.

Being new to TX2, I have dug into this forum looking for answers. I see a lot of references to “building the UVC” driver and gstreamer. Neither which I am familiar with nor have I found any clues on how to configure UVC and the build it.

If anyone can point me in the right direction I would be most appreciative.

Thanks,
Doug

Hi AeroClassics,
Looks like it is a v4l2 source. ARe you able to get information via v4l2-ctl?

v4l2-ctl -d /dev/video0 --list-formats-ext

@AeroClassics:

“I tried the camera and converter using VLC on my Ubuntu host machine”
I am just wondering what to execute to get at Host PC CSI video stream from network Jetson ?

Howerver, if your CSI device is not listed at

ls /dev/video*

, and it is not listed by default unless you load v4lloopback, as far as I know. For the purpose I am using

modprobe v4l2loopback devices=1 video_nr=2 exclusive_caps=1

that creates

/dev/video0

then I can use Bayer-type CSI at TX2 as UCV [generic usb camera type]

for reference: https://devtalk.nvidia.com/default/topic/1023416/?comment=5228435

I tried the v4l2-ctl command and it seems v4l2 is not installed on my TX2. Is this available using apt-get? I can tell you that when I connect this camera to the full size USB 3 port I do get a /dev/video1 device show up. Conversely, removing it cause the /dev/video1 device to be removed. So that is a positive indication I think.

Thanks,
Doug

You may install v4l-utils. This apt package contains the v4l2-ctl command.

sudo apt-get install v4l-utils

Thank you, the apt-get allowed me to run and gave me a nice list of formats:

ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘YUYV’
Name : YUYV 4:2:2
Size: Discrete 640x360
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 720x576
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 800x600
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 960x540
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1024x768
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1280x1024
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1600x1200
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1920x1200
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.017s (59.940 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.033s (29.970 fps)
Interval: Discrete 0.040s (25.000 fps)
Interval: Discrete 0.042s (24.000 fps)
Interval: Discrete 0.042s (23.976 fps)
Interval: Discrete 0.067s (15.000 fps)

This tells me that v4l2 can at least read something from the device. Encouraging! Now to figure out how to get gstreamer to work since that seems to show up in a lot of posts or v4l2. I ran the imagenet application and it opened the camera and successfully grabbed some frames and correctly classified what it was seeing (a dashboard). The I would see an error that it could not capture the frame and if failed to convert the frame from NV12 to RGBA. Why NV12 I am not sure since the input stream is YUV?

Since my ultimate goal here is to use OpenCV 3.4, Python and C++, do I need gstreamer or v4l2? Or are they called from within my application to do the necessary pipeline to get the video?

Thanks,
Doug

You may also install qv4l2:

sudo apt-get install qv4l2

It will give you easy access to different parameters for trying. Once you’ll have the right settings, set these with v4l-ctl.
Then you should be able to use gstreamer plugin v4l2src (8 bits modes only) with a pipeline such as:

gst-launch-1.0 v4l2src device=/dev/video1 ! 'video/x-raw,format=YUY2' ! xvimagesink

For opencv, I have not tried this format, but I have successfully received I420 frames in opencv3.3.
So a gstreamer pipeline is not mandatory, but if you want to use BGR format algorithms in opencv, you would have to convert. As far as I remember, I420->BGR was not available on gpu, only the CPU version was available, and this was just a bit slower than videoconvert…but I haven’t investigated much. It might have improved in opencv-3.4, but I don’t know.

Thank you! Much appreciated. I tried the gst-launch-1.0 for giggles and it runs for a minute or so then the display freezes. The TX2 isn’t locked up as you can close the windows or Ctrl-C in the terminal to stop gst-launch-1.0.

I have tried a variety of settings and it always stops. Sometimes I can get 2-300 and once in a while I can get upwards of 2000. The terminal only says ‘error’ and nothing more to give me a clue as to what might be happening. The program says the driver being used is the uvcvideo and the bus isusb-3530000.xhci-1.1. For card is does have the correct name of the converter. I just cannot seem to figure out why it quits.

Thoughts anyone?

Thanks,
Doug

Not sure it helps, but does this improve ? :

gst-launch-1.0 <b>-e</b> v4l2src device=/dev/video1 ! 'video/x-raw,format=YUY2' ! <b>queue !</b> xvimagesink <b>sync=false async=false</b>

If yes, you would have to find out which of these extra arguments makes the difference.

If not, you may activate gtsreamer debug such as:

<b>GST_DEBUG=*:3</b> gst-launch-1.0 -e<b>v</b> v4l2src device=/dev/video1 ! 'video/x-raw,format=YUY2' ! xvimagesink

Thank you! Looks like I am getting corrupted buffers. I was looking at the setup and I have a powered usb3 hub that I have the camera, keyboard and mouse connected to (just never enough USB ports). That got me to wondering. I unplugged the USB3 mini connecting my host that I SSH over and using the supplied adapter I connected the camera to the mini USB3 port on the TX2. Well, that certainly made a difference! The camera runs without a hitch in gstreamer or imagenet or detectnet. I ran q4l2 and found that that port is quite slow. IIf I set the size to 640 x 480 I get ~20 fps. If I set it to 1920 x 1080 the program select 5fps and you cannot change it!

Is the mini-USB port known to be especially slow? Or is there some setting I have not used?

Thanks again for everyone’s help!

Doug

The micro-USB port is wired only for USB2 or slower. It doesn’t have the wiring needed for USB3. You might consider a HUB on the micro port for keyboard, mouse, so on…and then only the camera on the full-sized port. Run “lsusb -t” to get a tree view…each node will be noted with its speed (USB2 is 480M, USB3 is 5000M…keyboard/mouse are only USB1.1 1.0).

@linuxdev, great answer! I will do that tomorrow. How will that affect my using the micro-USB port for SSH. I guess in the TX2 there is a mapping of that port to be IP Address 192.168.55.1. Both the host and the TX2 are connect to the outside world and have no domain name. Cheap and dirty for downloading things. Otherwise I get stuck behind a blasted proxy which a pain in the neck.

I do have a 10.10.10.XXX private net I could put both on and one of the other machines could act as a router, I suppose. My host is a NUM and only has 1 ethernet connection.

Doug

USB itself is not capable of being used for networking, though you might have a network device on USB. USB2 is capable of exceeding 100Mb/s ethernet, so this won’t be an issue for ssh (typically an interactive shell would use less than 115200bit/s). If you are using a gigabit dongle, then USB2 would limit speed (1000Mbit/s versus 0.1Mbit/s…gigabit is about 4 orders of magnitude faster than a fast terminal).

If you have just your video device on the USB3 port (meaning it is the only device on that root_hub), then you won’t have to worry so much about other USB devices competing for bandwidth (or perhaps more importantly, not having brief moments when the port is servicing something else). Do check with “lsusb -t” to see if it shows the desired 5000M for speed.

@linuxdev, quite right. I will rearrange things today and figure out a better LAN arrangement.

Thanks again everyone for all of your help. Been a while since I worked in this type of environment and it feels good to be back!

Doug