PointGrey Grasshopper Full Frame rate on TX1 [Solved]

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hi Folks,

I’m trying to get a point grey USB3 Grasshopper camera working on my TX1 Dev board.

I can currently get about 20-30 fps at full resolution (2048 x 1536). That translates to about 3MB per image, which suggests something like 60MB/s of transfer speed, so USB2 speeds. I’m running these tests from FlyCap2 (provided by Point Grey) and also from c++ through the point grey SDK. (I can post the code if that’s helpful, but it’s basically a stripped down version of this https://gist.github.com/kevinhughes27/5543668). Looking at TOP the CPU is at about 50-60% load, and memory usage is around 17-18%.

But, if I ask for a much smaller image size (say 320 x 240 = 0.07 MB per image), I STILL only get about 20-30 fps.

I followed the instructions to enable USB3 support, and that seems to have been successful based on this:

ubuntu@tegra-ubuntu:~$ lsusb -t
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
    |__ Port 2: Dev 4, If 0, Class=Miscellaneous Device, Driver=, 5000M
    |__ Port 2: Dev 4, If 1, Class=Miscellaneous Device, Driver=, 5000M
    |__ Port 2: Dev 4, If 2, Class=Miscellaneous Device, Driver=, 5000M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/5p, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=tegra-ehci/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M

Here’s my extlinux.conf

ubuntu@tegra-ubuntu:~$ cat /boot/extlinux/extlinux.conf 
TIMEOUT 30
DEFAULT primary

MENU TITLE p2371-2180 eMMC boot options

LABEL primary
      MENU LABEL primary kernel
      LINUX /boot/Image
      FDT /boot/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
      APPEND fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tegraid=21.1.2.0.0 ddr_die=2048M@2048M ddr_die=2048M@4096M section=256M memtype=0 vpr_resize usbcore.usbfs_memory_mb=10000 usbcore.autosuspend=-1 usb_port_owner_info=2 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=${lp0_vec} nvdumper_reserved=${nvdumper_reserved} core_edp_mv=1125 core_edp_ma=4000 gpt android.kerneltype=normal androidboot.touch_vendor_id=0 androidboot.touch_panel_id=63 androidboot.touch_feature=0 androidboot.bootreason=pmc:software_reset,pmic:0x0 root=/dev/mmcblk0p1 rw rootwait

I can plug the camera into my desktop and achieve the full 120fps at full resolution when plugged into a USB3 port. If I plug it into a USB2 port and ask for a 640 x 480 image, I can get 170fps on the desktop.

Any idea why I’m limited to 20-30 fps on the TX1? I’ve been focusing on the USB3 transfer speed, but I’m starting to think that’s not the issue. Is there a latency problem, or something else I’m not thinking about? Any help, suggestions, hints, wild creative fantastical theories much appreciated.

2

What is the output of:

cat /proc/cmdline

Which instructions did you use for setting USB3? The TX1 should work with “usb_port_owner_info=0” (you may have been following JTK1 instructions instead). Can you check “lsusb -t” again with “usb_port_owner_info=0”?

3

Thanks!

Here’s the output:

ubuntu@tegra-ubuntu:~/creare_data/git/LaserMetrologyCommon/Snap/env$ cat /proc/cmdline
fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tegraid=21.1.2.0.0 ddr_die=2048M@2048M ddr_die=2048M@4096M section=256M memtype=0 vpr_resize usbcore.usbfs_memory_mb=10000 usbcore.autosuspend=-1 usb_port_owner_info=2 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff2bf000 nvdumper_reserved=0xff23f000 core_edp_mv=1125 core_edp_ma=4000 gpt android.kerneltype=normal androidboot.touch_vendor_id=0 androidboot.touch_panel_id=63 androidboot.touch_feature=0 androidboot.bootreason=pmc:software_reset,pmic:0x0 root=/dev/mmcblk0p1 rw rootwait

I was following instructions from the JTK1, here (http://elinux.org/Jetson/Cameras#Enabling_support_for_USB_3.0_on_the_full-sized_USB_port).

Okay, I changed “usb_port_owner_info” from 2 → 0. Then I get:

ubuntu@tegra-ubuntu:~$ lsusb -t
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
    |__ Port 2: Dev 3, If 0, Class=Miscellaneous Device, Driver=, 5000M
    |__ Port 2: Dev 3, If 1, Class=Miscellaneous Device, Driver=, 5000M
    |__ Port 2: Dev 3, If 2, Class=Miscellaneous Device, Driver=, 5000M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/5p, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=tegra-ehci/1p, 480M
    |__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M

So the same thing. I don’t see a difference in my fps however.

4

The URL you mention is for the TK1, the TX1 does not need the USB port owner change (I think that when the article was written the TX1 was not yet available).

Was the “cmdline” from above before or after changing “usb_port_owner_info” to “=2”? I’m interested in knowing if “cmdline” shows “=2” both before and after the change to extlinux.conf’s usb_port_owner_info (and of course rebooting after change).

If “/proc/cmdline” follows what is in extlinux.conf for usb_port_owner_info after reboot, ignore what follows…

There is some possibility that even though the port is in USB3 mode that some parts of the system may still query the camera as if the bus is only USB2 for cases of “mistaken USB3/USB2 identity”. If cmdline follows what is in extlinux.conf that issue won’t happen (a case of each part of the chain of software using the same idea of being USB3 capable); if cmdline does not accurately reflect usb_port_owner_info, correct behavior depends on whether cmdline is used to base USB query decisions on.

Being a USB3 device capable of throttling back to USB2 on a USB2 mode root HUB would cause fewer responses of capabilities from a USB3 device, as intended…the device does not respond about USB3 capabilities if the device is never asked about USB3 capabilities the HUB can’t support. File “/proc/cmdline” is a reflection of what some drivers were told…should “/proc/cmdline” lie about the option being 0 versus 2, then drivers depending on this will believe cmdline told them. Other software may get their idea of USB2 versus USB3 from somewhere else (such as “lsusb -t” does) and behave correctly. Devices not capable of throttling back to USB2 standards might even cease to operate should cmdline tell them the port is only USB2.

I’m wondering if this is a case of some parts knowing the device is USB3, but other parts thinking the device is USB2.

5

Did you already try this ?
https://devtalk.nvidia.com/default/topic/919354/jetson-tx1/usb-3-transfer-failures/post/4899105/#4899105

6

@linuxdev

I the previous post the output for cat /proc/cmdline was before changing usb_port_owner_info.

So: when

  1. usb_port_owner_info=2 in extlinux.conf cat /proc/cmdline gives usb_port_owner_info=2
  2. usb_port_owner_info=0 in extlinux.conf cat /proc/cmdline gives usb_port_owner_info=0

Thanks for the idea though, I learned a lot from your writeup.

@sandmen yes, I’ve tried that, and no difference.

I saw that Jetpack got an update – I’m trying that now, and I"ll keep you posted.

7

Jetpack 2.2 did it for me. I can now get 100fps using the 64-bit Jetpack installation. Thanks for the help.

8

I am having the same problem. My USB3 Grasshopper is maxing out at about 45 frames per second at 1920x1200 resolution, instead of the rated 163 frames per second.

I used JetPack 3.0 to do a full install of Ubuntu 16.04 and it is running FlyCapture 2.11.3.121. The Grasshopper3 (GS3-U3-23S6M) is running firmware 2.22.3.0.

usb_port_owner_info=0 in extlinux.conf and I added usbcore.usbfs_memory_mb=1000 to the APPEND line.

It seems to be limited to USB2 bandwidth instead of the full USB3. Is there anything else I need to change to increase the framerate? Thanks for any advice you can give!

9

What is the output of “lsusb -t” and “sudo /home/ubuntu/jetson_clocks.sh --list”?

You may want to run:

sudo ./jetson_clocks.sh --store
sudo ./jetson_clocks.sh
# Compare to previous value from the --list run prior to --store:
sudo ./jetson_clocks.sh --list
# If you want to use the original settings the --store allows:
sudo ./jetson_clocks.sh --restore
10

ubuntu@tegra-ubuntu:~$ lsusb -t
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=tegra-ehci/1p, 480M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 2: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 2: Dev 4, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/4p, 5000M
|__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
|__ Port 2: Dev 3, If 0, Class=Miscellaneous Device, Driver=, 5000M
|__ Port 2: Dev 3, If 1, Class=Miscellaneous Device, Driver=, 5000M
|__ Port 2: Dev 3, If 2, Class=Miscellaneous Device, Driver=, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=tegra-xhci/5p, 480M

ubuntu@tegra-ubuntu:~$ sudo /home/ubuntu/jetson_clocks.sh --show
CPU frequency Governor: interactive
CPU MinFreq=102000 MaxFreq=1734000 CurrentFreq=1224000
CPU HOTPLUG: 0
Online CPUs: 0-3
cpu0: 1224000
cpu1: 1224000
cpu2: 1224000
cpu3: 1132800
CPU Cluster Switching: Disabled
GPU MinFreq=0 MaxFreq=998400000 CurrentFreq=76800000 FreqOverride=0
EMC MinFreq=0 MaxFreq=1600000000 CurrentFreq=1065600000 FreqOverride=0
Fan: speed=0
ubuntu@tegra-ubuntu:~$ sudo /home/ubuntu/jetson_clocks.sh --store
ubuntu@tegra-ubuntu:~$ sudo /home/ubuntu/jetson_clocks.sh
ubuntu@tegra-ubuntu:~$ sudo /home/ubuntu/jetson_clocks.sh --show
CPU frequency Governor: userspace
CPU MinFreq=102000 MaxFreq=1734000 CurrentFreq=1734000
CPU HOTPLUG: 0
Online CPUs: 0-3
cpu0: 1734000
cpu1: 1734000
cpu2: 1734000
cpu3: 1734000
CPU Cluster Switching: Disabled
GPU MinFreq=0 MaxFreq=998400000 CurrentFreq=998400000 FreqOverride=1
EMC MinFreq=0 MaxFreq=1600000000 CurrentFreq=1600000000 FreqOverride=1
Fan: speed=255

11

It looks like these are for the camera, and verifies that USB3 is running at its max speed:

|__ Port 2: Dev 3, If 0, Class=Miscellaneous Device, Driver=, 5000M
|__ Port 2: Dev 3, If 1, Class=Miscellaneous Device, Driver=, 5000M
|__ Port 2: Dev 3, If 2, Class=Miscellaneous Device, Driver=, 5000M

After running “sudo /home/ubuntu/jetson_clocks.sh” does your frame rate improve or change?

12

I did not see any improvement with the frame rate after running jetson_clocks. Perhaps there is problem with FlyCapture or how I am using it.

13

What mode are you using to capture? Free-running should give you the maximum frame rate. Any type of triggering can slow you down significantly unless you’re careful.

14

I’m not a camera guy, but with the system itself running at full speed the answer is likely as mentioned, related to the software or how the software is called.