Intel RealSense D435 USB3 Camera on TX2 - V4L2 Drops Frames at High Resolutions

Hi all,

Using either the built-in Jetson TX2 USB port or a PCIe uPD720202 based card, I can’t acquire at resolutions higher than 848x480 without having dropped frames with this camera. By dropped frames, I mean every second or third frame comes back smaller than expected, containing just a few lines of pixel data.

I’m testing using my own V4L2 application, but the same appears if I use yavta for example.

I should point out that the camera uses bulk mode and not isochronous mode:

lsusb -d '8086:0b07' -v | grep 'Transfer Type'
          Transfer Type            Interrupt
          Transfer Type            Bulk
          Transfer Type            Bulk
FIXME: alloc bigger buffer for device capability descriptors
          Transfer Type            Bulk

lsusb -t shows 5000M speed

/:  Bus 10.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 1: Dev 2, If 1, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 1: Dev 2, If 2, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 1: Dev 2, If 3, Class=Video, Driver=uvcvideo, 5000M
    |__ Port 1: Dev 2, If 4, Class=Video, Driver=uvcvideo, 5000M

Here is my yavta output:

./yavta --capture=20 --size 1920x1080 -t 1/30  /dev/video4
Device /dev/video4 opened: Intel(R) RealSense(TM) 435 (usb-0000:06:00.0-1).
Video format set: width: 1920 height: 1080 buffer size: 4147200
Video format: YUYV (56595559) 1920x1080
Current frame rate: 1/30
Setting frame rate to: 1/30
Frame rate set: 1/30
8 buffers requested.
length: 4147200 offset: 0
Buffer 0 mapped at address 0x7fa016f000.
length: 4147200 offset: 4149248
Buffer 1 mapped at address 0x7f9fd7a000.
length: 4147200 offset: 8298496
Buffer 2 mapped at address 0x7f9f985000.
length: 4147200 offset: 12447744
Buffer 3 mapped at address 0x7f9f590000.
length: 4147200 offset: 16596992
Buffer 4 mapped at address 0x7f9f19b000.
length: 4147200 offset: 20746240
Buffer 5 mapped at address 0x7f9eda6000.
length: 4147200 offset: 24895488
Buffer 6 mapped at address 0x7f9e9b1000.
length: 4147200 offset: 29044736
Buffer 7 mapped at address 0x7f9e5bc000.
0 (0) [-] 0 4147200 bytes 185437.733973 1542228849.119820
Warning: bytes used 65784 != image size 4147200
1 (1) [-] 1 65784 bytes 185437.739973 1542228849.120196
2 (2) [-] 2 4147200 bytes 185437.801172 1542228849.187045
Warning: bytes used 65784 != image size 4147200
3 (3) [-] 3 65784 bytes 185437.807220 1542228849.187409
4 (4) [-] 4 4147200 bytes 185437.868362 1542228849.254179
Warning: bytes used 65784 != image size 4147200
5 (5) [-] 5 65784 bytes 185437.874355 1542228849.254540
6 (6) [-] 6 4147200 bytes 185437.935557 1542228849.321373
Warning: bytes used 65784 != image size 4147200

Lower resolutions work fine. Anyone have any hints?

Can you please try ‘sudo ./jetson_clocks.sh’?

Also please try to run the sample tegra_multimedia_api\samples\12_camera_v4l2_cuda and check the result.

Thanks for the info, I will build and try this. I did run jetson_clocks.sh I should add, but it didn’t help.

Best,

  • Alex

I have tried the 12_camera_v4l2_cuda example by putting the stock JetPack L4T 3.2.1 image on my Jetson TX2, then applying a kernel patched to support the camera’s pixel formats in V4L2, and I ran the sample application.

When using resolutions of 848x480 or lower, the video display looked fine. But 1280x720 and 1920x1080 showed a flickering image - every second frame appeared ok, but the bad ones seemed to contain just a few rows of pixel information.

I tried to take a picture of it, but it doesn’t show up well on a cell phone camera.

This works fine on an Intel PC with Ubuntu 16.04

Hi precosky,
Can you try to connect D435 to USB3 HUB with external power supply and then connect the HUB to TX2 USB3 port?
Would like to check if is is an issue in power supply.

Besides, do you see any error/warning in dmesg?

Hi Dane,

I’ll get a powered hub (probably in the new week) and try this and I’ll update this once I’ve tried it.

In dmesg, I see just the following after issuing a command to yavta to acquire some images:

./yavta --capture=80 /dev/video2 -t 1/30 --size=1920x1080
[Nov23 20:21] xhci-tegra 3530000.xhci: exiting ELPG
[  +0.027320] xhci-tegra 3530000.xhci: Firmware timestamp: 2017-12-07 10:50:08 UTC, Version: 55.09 release
[  +0.013391] xhci-tegra 3530000.xhci: tegra_xhci_mbox_work mailbox command 6
[  +0.007157] xhci-tegra 3530000.xhci: exiting ELPG done
[  +5.183299] xhci-tegra 3530000.xhci: entering ELPG
[  +0.009272] xhci-tegra 3530000.xhci: entering ELPG done

The prints look normal. Please try powered hub.

Hi precosky,

Have you tried the powered hub? Any good news?

Thanks

Hi kayccc,

I have now acquired a powered hub, and plugged it in with its AC adaptor and connected the cable to the on-board USB port of the TX2 board. One realsense camrea is connected to the hub and nothing else.

Unfortunately the problem remains:

➜ dexr@precosky_tx2_a yavta git:(master) ✗ ./yavta /dev/video2 --capture=10 --size 1920x1080 -f YUYV -t 1/15
Device /dev/video2 opened: Intel® RealSense™ 435 (usb-3530000.xhci-1.3.1).
Video format set: width: 1920 height: 1080 buffer size: 4147200
Video format: YUYV (56595559) 1920x1080
Current frame rate: 1/30
Setting frame rate to: 1/15
Frame rate set: 1/15
8 buffers requested.
length: 4147200 offset: 0
Buffer 0 mapped at address 0x7f90622000.
length: 4147200 offset: 4149248
Buffer 1 mapped at address 0x7f9022d000.
length: 4147200 offset: 8298496
Buffer 2 mapped at address 0x7f8fe38000.
length: 4147200 offset: 12447744
Buffer 3 mapped at address 0x7f8fa43000.
length: 4147200 offset: 16596992
Buffer 4 mapped at address 0x7f8f64e000.
length: 4147200 offset: 20746240
Buffer 5 mapped at address 0x7f8f259000.
length: 4147200 offset: 24895488
Buffer 6 mapped at address 0x7f8ee64000.
length: 4147200 offset: 29044736
Buffer 7 mapped at address 0x7f8ea6f000.
0 (0) 0 4147200 bytes 3349.283954 1544117756.370382
Warning: bytes used 65784 != image size 4147200
1 (1) 1 65784 bytes 3349.296568 1544117756.371402
2 (2) 2 4147200 bytes 3349.418348 1544117756.504680
Warning: bytes used 65784 != image size 4147200
3 (3) 3 65784 bytes 3349.430931 1544117756.505247
4 (4) 4 4147200 bytes 3349.552707 1544117756.633103
Warning: bytes used 65784 != image size 4147200
5 (5) 5 65784 bytes 3349.559297 1544117756.633683
6 (6) 6 4147200 bytes 3349.687100 1544117756.766974
Warning: bytes used 65784 != image size 4147200
7 (7) 7 65784 bytes 3349.693162 1544117756.767392
8 (0) 8 4147200 bytes 3349.821518 1544117756.907716
Warning: bytes used 65784 != image size 4147200
9 (1) 9 65784 bytes 3349.833992 1544117756.908719
Captured 9 frames in 0.539176 seconds (16.692138 fps, 39068727.094678 B/s).

I’m getting support now through NVOnline but I’ll keep this updated if I learn anything.

Can you check the operating temperature of the device?

The cameras are just sitting on a desk. I’ve added temperature monitoring as per https://github.com/IntelRealSense/librealsense/issues/866#issuecomment-357461253
and reading a steady 28 deg C