Low frame rate with FLIR camera on TX2 when using cudafilters library from OpenCV

Hi,

Sorry for the late reply.

From your experiment, the issue may come from the camera → CUDA.
Do you use GStreamer for reading FLIR camera?

If yes, could you change the sample to read the video input via GStreamer?
This experiment can narrow down the issue is from FLIR camera or memory driver.

Thanks.

I’m using the FlyCapture library from FLIR to grab images but I don’t have access to the source code of the library. On the other side, I’m not aware of any way to grab images from FLIR camera using GStreamer. Is there any hypothesis that can be emitted on the memory driver ? Can I test such hypothesis ?

Hi,

We need to check the camera path first. Different route handles with a different driver.

Generally, CSI camera passes the nvcamerasrc component via GStreamer while web-camera use the v4l2src element.
Could you help us check which element the FLIR camera apply?

I’m not using GStreamer to capture the camera and I’m not aware of any way to do so. So I’m basically stuck with the library from the manufacturer.

Hi,

Another possibility may use the v4l2 interface to read the camera information into device DMA duffer.

Hi,
I’m out of the lab for the week. I will look into that at my return on oct. 2nd.

Hi francois,

It might be a long shot, but you FLIR USB camera might have some UVC driver compability. When you connect the camera, do you see something like this when running dmesg on the cmdline:

[12633.107599] uvcvideo: Found UVC 1.10 device LI-10635 (2a0b:00ea)
[12633.109205] input: LI-10635 as /devices/pci0000:00/0000:00:14.0/usb1/1-2/1-2.2/1-2.2:1.0/input/input23

dmesg gives this output after plugin the camera:

[ 87.721794] xhci-tegra 3530000.xhci: tegra_xhci_mbox_work mailbox command 4
[ 87.728827] xhci-tegra 3530000.xhci: tegra_xhci_mbox_work ignore firmware MBOX_CMD_INC_SSPI_CLOCK request
[ 87.939456] usb 2-1.4: new SuperSpeed USB device number 3 using xhci-tegra
[ 87.988625] usb 2-1.4: New USB device found, idVendor=1e10, idProduct=3300
[ 87.995565] usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 88.003033] usb 2-1.4: Product: Chameleon3 CM3-U3-13Y3C
[ 88.008338] usb 2-1.4: Manufacturer: Point Grey Research
[ 88.013749] usb 2-1.4: SerialNumber: 0105FB6C
[ 88.019110] xhci-tegra 3530000.xhci: tegra_xhci_mbox_work mailbox command 6

The camera is apparently not UVC compatible. I can see a /dev/video0 in my tree but can not grab from it with gstreamer.

This thread does not seem to have found a solution to directly grab the camera with gstreamer either:

https://devtalk.nvidia.com/default/topic/1001433/jetson-tx1/does-anyone-have-a-working-point-grey-camera-to-gstreamer-pipeline-/

From that dmesg you should be able to see the camera with “lsusb -d 1e10:3300”. If you look at a verbose listing of this it will tell you which parts are of a standard class and which parts require a custom driver:

lsusb -d 1e10:3300 -vvv

Here is the output from lsusb:

Bus 002 Device 004: ID 1e10:3300 Point Grey Research, Inc.
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2 ?
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 9
idVendor 0x1e10 Point Grey Research, Inc.
idProduct 0x3300
bcdDevice 0.00
iManufacturer 1 Point Grey Research
iProduct 2 Chameleon3 CM3-U3-13Y3C
iSerial 3 0105FB6C
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 116
bNumInterfaces 3
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 224mA
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 0
bInterfaceCount 3
bFunctionClass 239 Miscellaneous Device
bFunctionSubClass 5 USB3 Vision
bFunctionProtocol 0
iFunction 4 USB3 Vision Device
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 239 Miscellaneous Device
bInterfaceSubClass 5 USB3 Vision
bInterfaceProtocol 0
iInterface 0
** UNRECOGNIZED: 14 24 01 00 00 01 00 00 00 01 00 05 06 07 08 09 0a 0b 0c 0c
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 239 Miscellaneous Device
bInterfaceSubClass 5 USB3 Vision
bInterfaceProtocol 1
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 239 Miscellaneous Device
bInterfaceSubClass 5 USB3 Vision
bInterfaceProtocol 2
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
Binary Object Store Descriptor:
bLength 5
bDescriptorType 15
wTotalLength 22
bNumDeviceCaps 2
USB 2.0 Extension Device Capability:
bLength 7
bDescriptorType 16
bDevCapabilityType 2
bmAttributes 0x0000f61e
Link Power Management (LPM) Supported
SuperSpeed USB Device Capability:
bLength 10
bDescriptorType 16
bDevCapabilityType 3
bmAttributes 0x00
wSpeedsSupported 0x000c
Device can operate at High Speed (480Mbps)
Device can operate at SuperSpeed (5Gbps)
bFunctionalitySupport 2
Lowest fully-functional device speed is High Speed (480Mbps)
bU1DevExitLat 10 micro seconds
bU2DevExitLat 512 micro seconds
Device Status: 0x000c
(Bus Powered)
U1 Enabled
U2 Enabled

Can a USB3 vision device be used as a source in GStreamer ?

I see miscellaneous and bulk classes, so this is not a generic video device and probably requires some form of software to adapt it for use in video (you won’t automatically get a “/dev/video#” when this is connected to USB unless there is other software to act as a kind of “glue” to video). Is there any kind of downloadable software or requirements information with that camera?

UVC will transfer in isochronous mode (continuous real time), this camera transfers in bulk mode (like a hard drive dumping one picture at a time), plus control interface (the miscellaneous).

Hi francois.miralles,

Is there any update on this issue?
Thanks.